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MEASURING  NONCOMISSIONED  OFFICER  KNOWLEDGE  AND  EXPERIENCE  TO 
ENABLE  TAILORED  TRAINING 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  operational  tempo  of  the  United  States  Army  has  increased  dramatically,  placing  a 
premium  upon  effective  and  efficient  training.  However,  what  constitutes  effective  and/or 
efficient  training  varies  from  group  to  group  and  individual  to  individual.  For  decades 
researchers  have  explored  the  extent  to  which  training  quality  can  be  improved  by  tailoring 
training,  defined  as  assessing  salient  individual  differences  and  assigning  learners  to  learning 
conditions  based  on  those  differences.  Criterion-relevant  experience  and  prior  knowledge  are 
arguably  the  most  robust  predictors  of  performance,  and  are  thus  viable  candidates  as  bases  for 
tailoring  training.  However,  before  experience  and  prior  knowledge  can  be  used  to  make 
informed  decisions  about  tailoring  U.S.  Army  institutional  training,  effective  and  efficient 
measures  of  experience  and  prior  knowledge  must  be  developed  and  empirically  validated. 

Procedure: 

Instructors  from  the  Infantry  Advanced  Leaders  Course  (ALC)  at  Fort  Benning,  Georgia 
were  interviewed  to  determine  what  course  criteria  exhibited  large  variability  in  student 
performance.  Based  on  those  interviews,  two  criteria  were  chosen:  performance  on  a  land 
navigation  exercise  and  on  a  troop  leading  procedures  (TLP)  exam.  For  each  criterion,  four  types 
of  predictors  were  constructed.  The  first  predictor  type  consisted  of  small  group  instructor  (SGI) 
estimates  of  later  land  navigation  and  TLP  performance.  The  second  predictor  type  consisted  of 
general  biographic  items  which  anecdotal  evidence  indicates  instructors  use  to  assess  relevant 
experience.  The  third  type  asked  questions  related  to  criterion-relevant  activities  (e.g.,  familiarity 
with  computer  navigational  systems,  frequency  of  engagement  in  land  navigation  or  use  of  troop 
leading  procedures).  The  fourth  type  of  predictor  was  a  prior  knowledge  test.  The  instructors 
reviewed  the  instruments  and  recommended  revisions.  The  final  instruments  were  administered 
to  an  ALC  class  (N=  74). 

Findings: 

For  both  criteria,  prior  knowledge  alone  was  a  significant  predictor.  This  finding 
questions  the  utility  of  relying  upon  general  biographical  information  or  self-report  experience 
items  to  predict  performance.  Because  the  land  navigation  criterion  consisted  of  a  practical 
exercise  vice  the  paper-and-pencil  prior  knowledge  test,  the  predictive  validity  of  the  land 
navigation  prior  knowledge  test  was  statistically  significant  but  not  large.  However,  using  the 
seven  most  difficult  prior  knowledge  items,  we  were  able  to  predict  successful  performance  on 
the  land  navigation  criterion.  Because  the  correlation  between  the  TLP  prior  knowledge  test  and 
criterion  was  higher,  we  used  both  total  prior  knowledge  scores  and  a  subset  of  difficult  prior 
knowledge  items  to  predict  TLP  criterion  performance. 


v 


Utilization  and  Dissemination  of  Findings: 

The  findings  demonstrate  the  utility  of  prior  knowledge  measures  for  predicting 
performance  and  thus  infonning  subsequent  implementation  of  tailored  training.  These  findings 
have  been  disseminated  to  Infantry  ALC  instructors  at  Fort  Benning,  GA  and  briefed  to 
TRADOC  personnel  at  Fort  Eustis,  VA. 
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MEASURING  NONCOMISSIONED  OFFICER  KNOWLEDGE  AND  EXPERIENCE  TO 

ENABLE  TAILORED  TRAINING 

Introduction 

The  operational  tempo  of  the  U.S.  Army  has  increased  dramatically  in  recent  years. 
Soldiers  are  required  to  learn  more  in  less  time,  and  thereby  need  effective  and  efficient  training. 
However,  there  is  ample  evidence  both  that  learning  related  individual  differences  exist  (Jensen, 
1998;  Thorndike,  1985)  and  that  these  individual  differences  interact  with  learning  conditions 
(McNamara,  Kintsch,  Songer,  &  Kintsch,  1996).  Thus,  a  given  training  method  may  be  effective 
and  efficient  for  one  group  but  not  for  another  group.  For  decades  researchers  have  explored  the 
extent  to  which  training  quality  can  be  improved  by  tailoring  training,  defined  as  assessing 
salient  individual  differences  and  assigning  learners  to  learning  conditions  based  on  those 
differences. 

For  tailored  training  to  be  effective,  at  least  two  conditions  must  be  satisfied.  First,  there 
must  be  evidence  demonstrating  a  significant  relationship  between  one  or  more  individual 
differences  and  performance.  Second,  there  must  be  evidence  of  an  interaction  between  one  or 
more  individual  differences  and  the  training  condition  (Pashler,  McDaniel,  Doug,  &  Bjorn, 

2009).  The  goal  of  this  report  was  to  satisfy  the  first  condition:  to  isolate  individual  differences 
which  predicted  criteria  perfonnance  in  a  noncommissioned  officer  (NCO)  course. 

We  chose  to  focus  primarily  on  the  individual  difference  of  prior  knowledge  (defined  as 
information,  facts,  and  procedures  required  for  successful  performance  in  a  domain — see 
Palumbo,  Miller,  Shalin,  &  Steel-Johnson,  2005).  There  are  at  least  four  reasons  for  doing  this. 
First,  previous  research  which  based  predictor  selection  on  instructor  identification  of 
performance-relevant  individual  differences  was  not  successful  (Schaefer,  Bencaz,  Bush,  & 

Price,  2010),  suggesting  a  new  approach  was  needed.  Second,  measuring  prior  knowledge  is 
often  an  efficient  means  of  capturing  performance-relevant  variance.  Evidence  indicates  that 
general  mental  ability  is  the  most  robustly  predictive  of  broad  psychological  constructs  (Goska  & 
Ackerman,  1996;  Gottfredson,  1998;  Jensen,  1998;  Thorndike,  1985).  However,  general  mental 
ability  affects  perfonnance  through  the  acquisition  of  prior  knowledge.  In  addition,  experience 
(often  measured  simply  as  self-reported  length  of  time  working  in  a  given  domain)  also  affects 
performance  through  the  acquisition  of  prior  knowledge.  In  other  words,  general  mental  ability 
plus  experience  within  a  domain  contributes  to  prior  knowledge,  which  in  turn  contributes  to 
criterion  performance.  This  means  that  general  mental  ability  and  experience  significantly 
predict  prior  knowledge  but  not  criterion  performance.  Prior  knowledge,  as  the  variable  most 
directly  related  to  criterion  performance,  does  significantly  predict  criterion  performance.  Thus, 
measuring  prior  knowledge  captures  the  joint  effects  of  the  two  most  powerful  known  predictors 
of  performance  (Borman,  White,  Pulakos,  &  Oppler,  1991;  Borman,  White,  &  Dorsey,  1995; 
Palumbo,  Miller,  Shalin,  &  Steel-Johnson,  2005;  Schmidt  &  Hunter,  1993;  Schmidt,  Hunter,  & 
Outerbridge,  1986).  Third,  the  most  replicated  tailored  training  effects  involve  general  mental 
ability  and  prior  knowledge  (Kalyuga,  Ayres,  Chandler,  &  Sweller,  2003;  Snow,  1991,  1992). 
Fourth,  we  know  that  Soldiers  sometimes  vary  in  amount  of  prior  knowledge  (e.g.,  with  digital 
systems;  see  Bink,  Wampler,  Goodwin,  &  Dyer,  2008). 
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However,  we  did  not  rely  solely  on  measures  of  prior  knowledge.  We  also  constructed 
three  other  types  of  predictors.  First,  we  asked  small  group  instructors  (SGIs)  to  predict  the 
criterion  performance  of  the  Soldiers.  Second,  we  had  the  Soldiers  complete  a  demographic 
questionnaire  containing  general  infonnation  items  (e.g.,  military  occupational  specialty  or 
MOS,  deployment  experience,  etc.).  Third,  we  constructed  experience  scales  assessing  various 
aspects  of  specific,  criterion-related  activities. 

Our  rationale  behind  the  choice  of  these  predictor  types  was  as  follows.  First,  it  is  of 
obvious  interest  to  see  how  accurately  small  group  instructors  can  predict  Soldier  performance, 
as  presumably  adjustments  to  instruction  are  based  on  such  judgments.  Further,  research 
indicates  that  job  supervisors  appear  to  base  their  assessments  of  supervisee  job  perfonnance 
more  on  prior  knowledge  than  actual  job  performance  (Schmidt,  Hunter,  &  Outerbridge,  1986). 
In  other  words,  supervisor  perceptions  of  supervisee  performance  are  more  correlated  with  job 
knowledge  than  with  actual  job  perfonnance.  Including  instructor  predictive  judgments  allows 
us  to  see  if  a  similar  pattern  holds  in  a  military  course  with  SGIs.  Second,  anecdotal  evidence 
indicates  that  many  instructors  rely  upon  infonnal  cues  like  rank  and  deployment  experience  to 
make  predictions  about  Soldier  perfonnance.  Fonnally  assessing  the  predictive  power  of  such 
cues  via  the  demographic  questionnaire  allows  us  to  estimate  the  practical  utility  of  such 
information.  Third,  constructing  experience  scales  related  to  specific,  criterion-related  activities 
might  be  expected  to  yield  more  robust  prediction  than  less  targeted  predictors  like  MOS  or 
deployment.  Finally,  the  demographic  and  experience  scales  were  intended  to  address  a 
difference  between  the  occupational  literature  and  Army  institutional  settings.  In  the 
occupational  literature,  experience  is  measured  by  simply  asking  individuals  how  long  (e.g., 
months)  they  have  been  engaged  in  a  specific  domain  (Schmidt  &  Hunter,  1993;  Schmidt, 
Hunter,  &  Outerbridge,  1986).  Given  the  different  duty  assignments  performed  by  the  U.S. 
Soldier,  analogues  of  such  simple  measures  (e.g.,  time  in  grade  or  time  in  service)  were  judged 
unlikely  to  significantly  predict  criterion  perfonnance. 

Method 


Course  Selection 

Our  goal  was  to  identify  one  noncommissioned  officer  (NCO)  course  and  one  officer 
course  (Schaefer,  Blankenbeckler,  &  Lipinski,  2011)  with  Soldiers  who  varied  significantly  in 
performance-relevant  experience  and  knowledge.  To  guide  our  initial  selection  of  courses,  we 
developed  seven  criteria  (Appendix  A).  We  then  began  examining  courses  listed  in  the  Army’s 
Training  Requirements  and  Resource  System  (ATRRS)  to  identify  potential  courses.  At  the 
same  time,  we  developed  interview  protocols  for  use  with  course  personnel.  The  protocols  were 
designed  to  verify  information  obtained  in  ATRRS  as  well  as  gather  information  on  course 
prerequisites,  student  demographics,  and  the  nature  of  existing  course  perfonnance  criteria. 
Interviewing  instructors  from  the  potential  course  list  as  well  as  considering  the  availability  of 
course  personnel  during  the  research  timeframe  resulted  in  the  final  selection  of  the  Infantry 
Advanced  Leaders  Course  (ALC)  at  Fort  Benning,  Georgia. 

Description  of  Infantry  ALC.  The  Infantry  ALC  is  an  NCO  professional  development 
course  focused  on  training  Infantry  noncommissioned  officers,  primarily  staff  sergeants,  for  duty 
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as  squad  leaders.  Interviews  with  course  leaders,  staff,  and  small  group  instructors  (SGIs) 
indicated  that  Soldiers  arrive  at  the  course  from  diverse  backgrounds  and  with  widely  varied 
experiences  and  knowledge.  The  Infantry  ALC  is  a  single  gender,  male  only  course. 

There  are  two  versions  of  the  Infantry  ALC:  a  mobile  training  team  (MTT)  version  and  a 
resident  version.  The  MTT  version  involves  a  MTT  training  Soldiers  at  their  home  station. 

MTT  courses  are  typically  capped  at  65  Soldiers  located  at  the  given  home  station.  In 
comparison,  the  resident  version  of  Infantry  ALC  is  capped  at  160  students  and  is  open  to 
Soldiers  from  across  the  Army,  as  opposed  to  Soldiers  stationed  at  a  given  installation.  Both 
sample  size  considerations  and  selection  factors  led  us  to  the  conclusion  that  the  resident  version 
would  yield  larger  differences  in  relevant  experience  and  knowledge.  We  therefore  chose  the  4.5 
week-long  resident  version. 

Selection  of  Performance  Criteria 

Our  prior  research  focused  on  the  relationship  between  broad  cognitive  traits  (e.g., 
metacognition)  and  broad  measures  of  achievement  (e.g.,  overall  course  average — see  Schaefer, 
Bencaz,  Price,  &  Bush,  2010).  However,  using  narrower  criteria  makes  the  construction  of  prior 
knowledge  tests  more  tractable.  We  therefore  asked  instructors  about  narrower  perfonnance 
criteria  which  exhibited  large  performance  differences.  The  instructors  indicated  that  two  such 
areas  were  land  navigation  and  troop  leading  procedures  (TLP). 

Land  navigation.  Infantry  NCOs  are  expected  to  arrive  at  Infantry  ALC  proficient  in 
land  navigation.  However,  instructors  indicated  that  this  expectation  is  not  always  met.  Students 
are  tested  in  a  hands-on,  live  environment  using  a  map  and  compass  on  a  local  navigation  course. 
The  test  is  administered  in  the  first  week  of  the  course.  Instructors  indicated  that  no  go  (i.e.,  fail) 
rates  are  frequently  high,  and  that  this  is  in  part  because  land  navigation  is  often  achieved  by 
using  Global  Positioning  System  (GPS)  devices  rather  than  map  and  compass.  The  test  consists 
of  8  terrain  points  which  students  must  locate  within  three  hours,  and  thus  possible  scores  range 
from  0  to  8.  A  ‘Go’  is  defined  as  locating  5  or  more  terrain  points.  The  exercise  is  five  hours 
long  and  begins  two  hours  before  dawn. 

TLP.  Instructors  also  indicated  that  many  students  arrive  with  only  limited  experience  in 
preparing  operations  orders,  operations  overlays,  or  conducting  TLP.  All  of  these  skills  in 
various  combinations  are  tested  on  the  TLP  test,  which  takes  place  approximately  2  weeks  into 
the  course.  The  TLP  exam  is  a  30-item,  multiple  choice/fill-in-the-blank  test  assessing  the  skills 
of  conducting  reconnaissance,  preparing  an  operational  overlay,  and  conducting  TLP.  ‘Go’ 
status  is  achieved  by  correctly  answering  70  percent  or  more  (i.e.,  21  or  more)  of  the  test  items. 

Participants 

Participating  in  this  research  were  74  Infantry  ALC  Soldiers  and  6  SGIs.  All  of  the 
Infantry  ALC  solders  were  1  IB  infantry  and  all  but  one  individual  was  a  Staff  Sergeant. 

Average  time  in  grade  in  months  (M=  20.57,  SD  =  10. 10)  and  time  in  service  ( M=  88.56,  SD  = 
30.37)  was  slightly  less  than  two  and  slightly  more  than  seven  years,  respectively. 
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All  resident  course  SGIs  were  contractors  and  former  NCOs  with  experience  in 
instructing  at  least  2  iterations  of  the  course.  While  there  was  a  potential  upper  bound  of  160  for 
this  sample,  inclement  weather  and  attendant  disruptions  to  plane  travel  reduced  the  number  of 
available  students.  This  had  a  large  impact  because  the  predictors  were  administered  on  the  first 
day  of  the  course. 

Procedure 

An  initial  group  interview  was  held  with  the  Infantry  ALC  instructors.  We  explained  that 
our  research  goal  was  to  target  one  or  more  performance  criteria  in  the  course  which  exhibited 
large  differences  in  student  perfonnance.  During  the  interview,  instructors  indicated  their 
confidence  that  early  in  the  course  they  could  predict  which  students  would  and  would  not  do 
well  on  the  land  navigation  and  TLP  criteria. 

After  the  criteria  were  chosen,  we  developed  initial  drafts  of  the  predictor  measures  and 
submitted  them  to  ALC  instructors  for  review.  In  general,  the  feedback  on  the  measures  was 
positive,  with  the  exception  of  some  questions  on  the  land  navigation  prior  knowledge  test. 
Instructors  expressed  concern  that  some  of  the  items  were  very  hard  and  suggested  that  the  hard 
items  be  placed  at  the  end  of  the  test  to  prevent  discouraging  students  from  completing  the 
measure.  The  instructors  also  suggested  that  we  indicate  that  solving  the  difficult  items  did  not 
require  a  map.  Once  these  changes  were  made,  all  measures  were  approved. 

The  student  measures  were  administered  to  the  Infantry  ALC  class  on  the  first  day  of 
class,  and  took  between  one  and  two  hours  to  complete.  Participating  SGIs  supplied  their 
predictions  of  student  criterion  performance  on  the  fourth  day  of  class,  and  the  land  navigation 
exercise  took  place  on  the  fifth  day  of  class.  The  TLP  exam  took  place  on  the  ninth  day  of  class. 
Notably,  we  were  not  permitted  to  examine  the  TLP  criterion  ahead  of  time,  but  were  provided 
access  to  the  tests  and  scores  later.  We  were  given  access  not  only  to  the  test  summary  scores 
but  also  the  test  items,  and  hence  were  able  to  verify  that  the  skills  tapped  by  the  prior 
knowledge  test  and  criterion  exam  overlapped.  If  we  had  been  able  to  view  the  criterion 
beforehand  and  validate  our  selection  and  sampling  of  skills  on  the  prior  knowledge  test,  we 
might  have  been  able  to  ensure  greater  skill  overlap  between  predictor  and  criterion. 

Measures 

SGI  predictions.  SGIs  were  asked  to  predict  how  students  would  perform  on  the 
criteria.  To  make  prediction  tractable,  we  did  not  ask  instructors  to  rank  order  the  students  from 
absolute  highest  to  lowest.  Instead,  we  asked  them  to  indicate  those  students  which  they  felt 
would  fall  into  the  bottom  25%,  the  middle  50%,  or  the  top  25%  of  the  criterion  distributions 
(Appendix  B).  Instructors  were  first  asked  to  predict  the  performance  of  students  in  their  group, 
and  were  also  given  the  further  option  of  predicting  the  performance  of  other  students  (i.e., 
outside  their  group). 

Demographic  questionnaire  and  experience  scales.  Students  first  read  a  statement  of 
informed  consent  and  then  completed  a  demographic  questionnaire  and  various  experience  scales 
(see  Appendix  C  for  measures  and  response  frequency  information).  Selection  of  demographic 
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items  was  guided  by  several  considerations.  For  example,  instructor  interviews  indicated  that 
information  such  as  Soldier  MOS  and  deployment  was  used  to  predict  student  performance.  In 
addition,  our  prior  research  has  found  that  level  of  education  can  affect  predictor-criterion 
relationships  (Schaefer,  Bencaz,  Bush,  &  Price,  2010). 

The  rationale  underlying  the  use  of  experience  scales  was  as  follows.  Schmidt,  Hunter, 
and  Outerbridge  (1986)  found  that  a  simple  index  of  experience  (e.g.,  total  months/years)  within 
a  domain  predicted  prior  knowledge.  Given  the  fact  that  Soldiers  may  fill  many  duty  positions 
during  their  service,  it  is  difficult  to  construct  such  a  direct,  simple  question  regarding  criterion¬ 
relevant  experience.  However,  because  asking  questions  related  to  time  in  grade  and  time  in 
service  require  little  effort,  we  included  such  items  in  the  demographic  questionnaire. 

Nonetheless,  just  as  prior  knowledge  is  a  more  targeted  and  therefore  more  powerful 
predictor  of  criterion  performance  than  general  mental  ability,  perhaps  asking  targeted  (i.e., 
specifically  criterion-related)  experience  questions  would  also  prove  fruitful.  We  developed  the 
experience  scale  questions  by  tapping  infantry  skills  associated  with  land  navigation  and  TLP. 
Only  two  of  the  experience  questions  (Questions  10  and  1 1)  asked  for  simple  yes/no  responses. 

All  other  experience  scale  items  elicited  self-assessments  via  one  of  two  methods.  First, 
the  Soldiers  were  asked  to  self-report  how  often  they  had  engaged  in  either  land  navigation 
(Questions  12  to  20)  or  TLP  (Questions  23  to  29  and  30  to  32)  activities.  Second,  the  Soldiers 
were  provided  with  brief  tactical  vignettes  and  asked  to  self-rate  their  land  navigation  (Questions 
2 1  and  22)  skills  based  on  a  series  of  graduated  actions  associated  with  the  situation. 

Prior  knowledge  tests.  The  prior  knowledge  tests  (Appendices  D  and  E)  were  designed 
to  assess  the  standing  of  Soldiers  with  regard  to  skills  which  the  course  either  presupposed  they 
already  possessed  (as  with  land  navigation)  or  built  upon  (as  with  TLP).  An  interplay  of  the 
following  four  factors  drove  the  construction  of  both  tests. 

First,  military  subject  matter  expertise  guided  the  construction  of  items  judged  to  be 
easier  and  more  difficult  in  the  given  domains  (land  navigation  or  TLP).  Second,  the  tests  did 
not  rely  upon  the  ability  to  recall  or  list  facts  and  terms,  but  to  use  information  and  apply 
principles  in  the  correct  manner.  This  would  also  serve,  presumably,  to  highlight  differences  in 
conceptual  understanding  which  might  not  be  brought  out  by  simple  recall.  Third,  the  tests  were 
designed  to  provide  a  measure  of  student  knowledge  without  additional  resources.  For  example, 
on  the  TLP  prior  knowledge  test,  images,  maps,  and  a  list  of  the  TLP  procedures  were  provided. 
Further,  on  the  land  navigation  test  questions  were  developed  to  avoid  the  need  for  a  compass  or 
protractor.  This  obviously  aids  in  efficiency,  as  the  tests  become  a  stand-alone  instrument.  The 
real  goal,  however,  was  to  see  if  Soldiers  could  rely  on  their  conceptual  understanding  of  what  a 
compass  or  protractor  helps  them  do  and  replicate  that  procedure  mentally.  This  can  be 
illustrated  by  highlighting  the  exchange  in  which  instructors  asked  us  to  place  those  land 
navigation  questions  they  perceived  as  harder  at  the  end  of  the  test  and  also  asking  us  to  make  it 
clear  that  those  questions  can  be  solved  without  a  map.  Fourth,  the  questions  on  the  tests  were 
designed  to  prevent  easy  discrimination  between  correct  and  incorrect  responses.  This  was 
accomplished  by  including  common  errors  as  options,  or  providing  them  with  an  item  which  did 
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not  correspond  to  the  options  given  in  the  question.  (See  Question  2  in  Appendix  D  for  an 
example  of  this.) 

For  the  land  navigation  test  in  particular  (Appendix  D),  sections  of  varied  maps,  overhead 
photographs,  varied  locations  on  the  maps  and  photos,  and  ground-based  photographic  images 
were  used  to  provide  clear,  unambiguous  examples  of  terrain  features,  topographic  map  symbols, 
and  comparisons  of  locations  as  the  basis  for  questions.  This  required  the  Soldier  to  apply  map¬ 
reading  skills,  not  simply  recall  infonnation  broadly  related  to  map-reading. 

The  questions  were  derived  from  a  variety  of  skills  required  for  night  and  day  land 
navigation,  both  of  which  were  tapped  by  the  land  navigation  criterion  practical  exercise.  Skills 
included  plotting  grid  coordinates,  knowing  cardinal  directions,  understanding  map  features  and 
colors,  measuring  distance,  recognizing  map  scales,  and  using  a  compass  at  night.  The  intent 
was  to  evaluate  knowledge  that  was  directly  applicable  and  necessary  to  successfully  perform  the 
practical  exercise. 

Questions  also  addressed  advanced  orienteering  skills  which  required  an  understanding  of 
standard  military  topographic  map  design  and  basic  geometry  above  and  beyond  the  Skill  Level 
1  land  navigation  and  map  reading  skills  tapped  by  the  practical  exercise.  The  intent  was  to 
identify  Soldiers  with  map  reading  and  land  navigation  skills  surpassing  the  requirements  of  the 
practical  exercise.  The  underlying  assumption  was  that  Soldiers  who  possessed  more  than  basic 
skills  would  be  quite  likely  to  do  well  in  the  practical  exercise. 

Questions  used  varied  locations  and  situations  so  that  each  question  stood  alone.  Some 
questions  were  brief  and  relied  on  graphics,  while  others  required  development  of  a  context  and 
required  more  reading  and  comprehension.  Creating  each  question  as  a  stand-alone  entity 
precluded  the  student  from  missing  a  subsequent  question  that  built  on  a  previous  situation.  This 
also  avoided  requiring  that  the  questions  be  answered  in  a  fixed  sequence. 

In  discussing  the  land  navigation  prior  knowledge  test,  it  must  be  noted  that  the  predictor 
was  paper-and-pencil,  yet  the  criterion  was  a  hands-on  practical  exercise.  Based  on  the  research 
literature  (Dubois  &  Shalin,  2005),  we  knew  that  the  overall  correlation  between  the  predictor 
and  criterion  would  likely  be  much  lower  than  that  for  TLP.  However,  for  several  reasons  we 
judged  this  a  worthwhile  line  of  inquiry.  First,  we  were  interested  in  establishing  a  quick  and 
easily-administered  predictor  of  performance.  Second,  there  are  costs  associated  with 
conducting  field  exercises.  In  the  Infantry  ALC,  land  navigation  is  not  part  of  the  fonnal 
program  of  instruction  but  included  to  address  a  perceived  gap  in  training.  Thus,  Infantry  ALC 
SGIs  may  be  able  to  opt  out  certain  students  from  formal  assessment  via  the  land  navigation 
practical  exercise.  Third,  there  is  little  systematic  examination  of  hands  on  (vice  cognitive)  tasks 
in  the  research  literature. 

For  the  TLP  test  (Appendix  E),  the  questions  assessed  the  use  of  TLP  in  planning/preparing 
for  an  operation,  understanding  a  standard  operations  order,  and  interpretation  of  an  operation’s 
overlay  using  standard  military  graphics.  The  Soldier  was  provided  with  the  steps  of  TLP  and 
questions  related  to  doctrinal  uses  of  the  process  in  planning  and  preparing  for  operations.  For 
the  operations  order  assessment,  the  Soldier  was  provided  with  the  format  for  a  standard  five- 
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paragraph  field  order.  The  vignette  and  questions  required  the  Soldier  to  organize  the 
information  received  from  higher  headquarters  into  his  own  operations  order  as  notes  for  issuing 
an  oral  order  to  his  squad.  For  assessment  of  military  graphics  knowledge  and  skills,  a  diagram 
was  provided  supplemental  to  the  photomap.  Both  provided  friendly  and  enemy  information  and 
the  overlays  used  standard  military  graphic  symbols.  Questions  required  the  Soldier  to  identify 
symbols  and  interpret  their  meaning  in  context  of  the  situation.  For  example,  the  Soldier  was 
required  to  identify  the  location  of  specific  threats  to  his  unit  and  mission,  as  well  as 
responsibilities  within  defined  sectors  and  unit  boundaries. 

Section  I  of  the  test  (questions  1-4)  used  a  brief  scenario  to  focus  the  NCO  on  a  mission 
requiring  preparation  for  offensive  action.  The  intent  was  to  have  the  student  demonstrate  an 
understanding  of  TLP  in  the  context  of  how  TLP  would  be  applied  and  used  by  the  squad  leader 
in  a  typical  duty  position.  The  steps  of  TLP  were  provided  and  the  student  was  required  to 
demonstrate  an  understanding  of  the  process.  Multiple  response  questions  were  employed  to 
provide  insight  to  the  student’s  understanding  of  aspects  of  TLP  that  were  continuous  or  that 
could  occur  more  than  once  in  the  process.  The  varied  purposes  and  uses  of  a  warning  order 
(WARNO)  were  also  the  subject  of  a  question. 

Section  II  of  the  test  (questions  5-13)  placed  the  student  in  the  role  of  a  squad  leader 
preparing  for  a  mission.  The  squad  leader  was  to  prepare  to  brief  his  squad.  This  is  a  task  that 
squad  leaders  would  normally  perform  in  the  line  of  their  duties.  Details  from  the  platoon 
leader’s  order  and  the  format  of  a  five-paragraph  field  order  were  provided,  which  is  the 
information  the  squad  leader  would  normally  have  available.  The  task  was  to  organize  the 
information  to  brief  the  order  to  the  team  leaders  or  squad.  The  students  were  required  to  make 
decisions  about  organizing  the  available  information  to  facilitate  the  mission  brief  to  the  squad. 
Some  latitude  was  provided  for  student  responses. 

Section  III  of  the  test  (questions  14  -  20)  focused  on  the  student’s  ability  to  understand  and 
interpret  military  symbols  and  graphics  common  to  company  and  platoon  operations  overlays 
and  digital  system  displays  (e.g.,  Force  XXI  Battle  Command  Brigade  and  Below  [FBCB2]).  A 
photo  map  and  scaled  sketch  of  a  battalion  operation  in  an  urban  area  sector  were  provided.  This 
is  typical  infonnation  the  squad  leader  would  normally  have  available  when  performing  his 
duties.  Questions  focused  on  understanding  the  symbols,  the  threat,  and  unit  responsibilities  in 
order  to  apply  this  infonnation  in  perfonning  as  a  squad  leader. 

Analysis  Strategy 

All  analyses  were  conducted  on  SPSS  16.0  for  Windows,  and  the  alpha  level  for 
significance  set  at  .05  for  all  tests.  As  this  was  an  exploratory  analysis,  all  p  values  should  be 
treated  with  caution.  We  reported  p  values  for  the  sake  of  completeness,  but  did  not  adjust  for 
family  wise  error  rate.  Any  confidence  in  the  strength  or  pattern  of  the  relationships  should  be 
tempered  in  the  absence  of  replication.  In  analyzing  the  data,  we  used  the  following  3-stage 
strategy. 

Data  screening  and  scale  construction.  First,  all  predictor  variables  were  examined  for 
differential  response  rates  (operationally  defined  as  any  question  or  item  to  which  more  than  80 
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percent  of  responses  fell  into  a  single  category  or  assumed  a  single  value),  truncation  of  range, 
many  response  categories  with  few  individuals,  and  other  issues.  If  any  such  problems  were 
found  with  an  item,  the  item  was  dropped  from  further  analysis  and  a  rationale  for  the  decision 
given.  Second,  all  experience  measure  items  were  grouped  into  scales  whenever  possible.  This 
was  done  by  first  examining  the  individual  question  descriptives.  If  no  problems  were  found, 
then  questions  were  grouped  on  the  basis  of  common  content  and  format.  Cronbach’s  alphas 
were  then  computed  to  assess  scale  reliability.  Unless  removing  an  item  resulted  in  an 
improvement  in  the  scale’s  Cronbach’s  alpha  by  .10  or  more  (e.g.,  the  scales  Cronbach’s  alpha 
would  increase  from  .80  to  .90),  all  scales  were  left  intact.  Any  potential  scale  items  which 
exhibited  no  item-level  statistical  problems  but  which  were  insufficiently  reliable  when  made 
into  a  scale  were  retained  as  stand-alone  predictors.  For  example,  say  we  had  ten  questions 
which  displayed  no  statistical  problems  individual.  However,  only  eight  of  the  questions 
cohered  into  a  consistent  scale.  Then  the  two  ‘excluded’  questions  would  be  retained  as 
individual  predictors  in  any  further  correlation  and/or  regression  analyses. 

Correlation  and  regression.  In  this  second  stage,  all  variables  retained  from  the  first 
stage  were  entered  into  a  correlation  matrix  with  the  relevant  criterion.  Based  on  the  Schmidt, 
Hunter,  &  Outerbridge  (1986)  findings,  we  had  three  expectations  of  the  data.  First,  we  expected 
that  prior  knowledge  would  significantly  predict  criterion  performance,  and  that  it  would  in  fact 
be  the  strongest  predictor.  Second,  we  expected  that  one  or  more  of  the  experience  variables 
(time  in  grade,  time  in  service,  experience  scales,  and  demographic  variables)  would 
significantly  predict  prior  knowledge,  but  would  not  predict  criterion  performance.  Third,  we 
expected  that  the  SGI  predictions  would  significantly  predict  prior  knowledge,  but  not  criterion 
performance.  These  expectations,  if  met,  would  argue  for  using  prior  knowledge — not 
experience  or  proxies  of  experience  like  the  included  demographic  items — to  predict  criterion 
performance.  This  is  because,  as  noted  in  the  introduction  section,  the  Hunter,  Schmidt,  and 
Outerbridge  (1986)  findings  indicate  that  experience  is  indirectly  related  to  criterion 
performance.  The  experience-criterion  relationship  is  therefore  too  weak  to  serve  as  the  basis  for 
making  tailored  training  decisions. 

If  more  than  one  significant  predictor  was  found,  both  simultaneous  and  stepwise 
regressions  were  computed.  Simultaneous  regression  gives  an  estimate  of  the  upper  limits  of 
predictability,  while  stepwise  regression  estimates  the  utility  of  using  only  a  subset  of  predictors. 
This  is  useful  information,  as  combining  infonnation  from  multiple  predictors  is  easy  when  using 
statistical  software  but  might  be  burdensome  for  the  envisioned  ‘end  user’  who  is  unlikely  to 
have  access  to  such  software.  A  more  sensible  procedure  might  be  for  the  end  user  to  focus  on 
two  (or,  better,  one)  robust  predictor  of  criterion  performance. 

Predicted  versus  observed  performance  categories.  This  third  and  final  stage  focused 
on  illustrating  how  different  predictors,  with  different  perfonnance  criteria  (i.e.,  hands-on  land 
navigation  practical  exercise  vice  paper-and-pencil  TLP  exam)  could  be  translated  into  ‘user 
friendly’  infonnation  for  use  by  course  instructors,  managers,  and  other  relevant  personnel.  We 
approached  this  problem  in  the  following  way. 

We  followed  Cohen’s  (1992)  proposed  lower  boundary  for  a  large  effect  size  as  a 
conelation  of  .37  or  larger.  If  such  a  conelation  was  found,  we  then  subjected  the  variables  to 
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both  Steps  1  and  2  (outlined  below).  If  such  a  correlation  was  not  found,  we  skipped  Step  1  and 
proceeded  to  Step  2. 

Step  1:  Total  score  relationships.  For  these  procedures,  we  visually  scanned  the 
predictor  and  criterion  total  score  frequency  distributions  to  see  if  naturally  occurring  break 
points  were  present.  To  foreshadow  our  results,  we  found  that  the  TLP  predictor  and  criterion 
distributions  both  suggested  a  break  into  thirds.  (Obviously,  different  break  points  might  be 
constructed  on  the  basis  of  instructor  judgment.  For  example,  an  instructor  might  be  interested 
in  the  top  and  bottom  10  percent.)  We  then  examined  the  relationship  between  the  predictor  and 
criterion  thirds  by  constructing  crosstabs  indicating  the  number  of  Soldiers  who  were  correctly 
(and  incorrectly)  classified  on  the  basis  of  their  standing  on  the  predictor  variable.  We  then 
repeated  the  crosstab  procedure,  but  this  time  compared  the  relationship  between  predictor  thirds 
and  Go/No  Go  status  on  the  criterion.  This  allows  for  the  comparison  of  predictor  standing  with 
a  criterion  that  would  be  readily  familiar  to  instructors. 

Step  2:  Subsets  of  easy  and  hard  prior  knowledge  items.  For  all  predictor/criterion 
pairs  we  attempted  to  isolate  subsets  of  the  easiest  and  hardest  prior  knowledge  items  and 
assessed  their  relationship  to  total  criterion  scores.  First,  crosstabs  between  the  easiest  and 
hardest  prior  knowledge  items  and  criterion  scores  were  created  to  see  if  interpretable  patterns 
emerged.  Second,  crosstabs  were  created  to  see  if  there  was  any  evidence  of  an  interpretable 
relationship  between  easy/hard  item  performance  and  Go/No  Go  status  on  the  criterion. 

As  discussed  earlier,  our  rationale  for  this  approach  was  that  individuals  who  fared  poorly 
on  the  easy  prior  knowledge  items  would  likely  fare  poorly  on  the  land  navigation  criterion,  and 
individuals  who  fared  well  on  the  hard  prior  knowledge  items  would  fare  well  on  the  criterion. 
Such  ‘mini  tests’  might  save  course  personnel  the  time  of  scoring  the  full  test  (although,  as  we 
explain  later,  there  might  be  reasons  to  administer  the  complete  prior  knowledge  test). 

Results 

To  improve  readability,  a  minimum  of  statistics  is  cited  in  the  text.  In  the  case  of  more 
complex  response  patterns,  a  verbal  summary  is  provided.  When  the  phrase  ‘most  respondents’ 
is  used,  this  means  that  more  than  80%  of  Soldiers  gave  the  same  response,  and  that  the  item  was 
dropped  from  further  analysis.  (See  Appendix  F  for  descriptive  statistics  of  the  retained 
variables.) 

Data  Screening  and  Scale  Constructions 

All  variables  were  examined  both  descriptively  and  graphically.  When  a  decision  was 
made  to  exclude  a  variable  from  further  analysis,  a  reason  for  that  exclusion  was  given. 

Variables  were  examined  in  the  order  in  which  they  appeared  in  the  Appendices  and  in  which 
they  were  described  above  (e.g.,  SGI  predictions,  demographic  questions,  experience  scales, 
prior  knowledge  tests,  and  performance  criteria).  When  we  reported  criterion  statistics,  we  did 
so  both  for  total  points  and  percent  correct. 
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SGI  predictions.  Although  most  SGIs  indicated  during  the  interviews  their  belief  that 
they  can  intuitively  assess  current  experience  and  knowledge  as  well  as  predict  future 
performance,  many  were  reluctant  to  make  formal  assessments  when  requested.  Further,  despite 
initial  confidence  that  accurate  intuitive  prediction  was  possible  early  on,  instructors  felt  that 
they  did  not  have  sufficient  time  with  the  students  to  fonn  an  accurate  opinion.  The  result  for 
both  criteria  was  that  fewer  than  50%  of  all  Soldiers  had  SGI  predictions.  Therefore,  this 
variable  was  dropped  from  further  analysis.  This  also  means  that  the  three  ‘expectations’  of  the 
data  were  reduced  to  two. 

Demographic  questionnaire.  All  items  in  the  demographic  questionnaire  were 
examined  for  problems.  As  stated  above,  decisions  to  exclude  variables  from  subsequent 
analyses  are  clearly  stated  and  defended.  Because  many  of  the  demographic  variables  were 
dropped  in  this  stage  of  analysis,  we  grouped  the  variables  into  ‘dropped’  and  ‘retained’. 

Dropped  variables.  There  were  two  factors  which  caused  demographic  items  to  be 
dropped  from  further  analysis.  Most  respondents  answered  the  rank,  military  occupational 
specialty  (MOS),  and  service  status  questions  the  same  way,  so  these  variables  were  dropped. 

The  second  factor  was  too  many  response  categories  with  too  few  responses  in  each  category. 
This  led  to  the  exclusion  of  questions  asking  year  of  Warrior  Leader  Course  (WLC)  or  Primary 
Noncommissioned  Officer  Course  (PNCOC)  completion  as  well  as  the  deployment  questions, 
replicating  a  pattern  seen  in  our  earlier  research  (Schaefer,  Bencaz,  Price,  &  Bush,  2010). 

It  is  worth  noting  that  this  does  not  preclude  this  information  being  correctly  perceived  as 
useful  by  instructors.  For  example,  prior  iterations  of  the  course  could  have  broken  along 
cleaner  lines  with,  say,  half  of  the  class  having  a  specific  deployment  experience  and  the  other 
half  not.  Such  a  pattern  would  lend  itself  both  to  instructor  perception  and  statistical  analysis  of 
predictor/criterion  relationships.  However,  the  data  we  do  have  (combined  with  prior  research, 
as  noted  above)  does  not  engender  confidence  in  using  such  items  to  make  tailored  training 
decisions. 

Retained  variables.  The  retained  demographic  variables  were  time  in  grade,  time  in 
service,  and  civilian  education  levels.  Although  the  time  in  grade  and  service  questions  were 
posed  in  terms  of  years  and  months,  for  ease  of  computation  we  translated  both  of  these  variables 
into  total  months.  Responses  to  the  civilian  education  level  ranged  from  GED  to  Bachelor’s 
degree,  with  most  responses  falling  into  the  ‘high  school  diploma’  or  ‘some  college’  options.  As 
there  were  so  few  individuals  with  college  degrees,  however,  the  follow-on  question  asking  for 
the  specific  field  of  study  (e.g.,  Business,  Computer  Science)  was  dropped. 

Experience  scales. 

Land  navigation.  The  first  potential  experience  scale  for  land  navigation  involved 
Questions  10a,  10b,  10c,  and  11.  All  asked  if  students  had  experience  with  digital  systems  for 
navigational  support  and  opera tions/situation  awareness  support.  Unfortunately,  most 
respondents  answered  Questions  10a,  10b,  and  1 1  the  same  way,  so  these  variables  were 
dropped.  This  left  Question  10c  on  use  of  maps  and  compass  as  primary  navigation  aids  as  a 
predictor.  (See  Appendix  D  for  the  specific  questions.) 
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The  second  potential  experience  scale  for  land  navigation  involved  Questions  12  through 
20,  which  asked  students  to  estimate  how  frequently  they  had  engaged  in  land  navigation  related 
behaviors.  Possible  responses  ranged  from  “never”  to  “daily.”  Descriptive  analyses  of  these 
questions  revealed  no  item-level  problems,  and  Cronbach’s  alpha  was  .92.  This  scale 
(henceforth  the  Land  Navigation  Frequency,  or  LNF,  Scale)  was  retained. 

The  third  potential  experience  scale  for  land  navigation  involved  Questions  2 1  and  22, 
which  placed  students  in  a  vignette.  Both  of  the  questions  exhibited  no  statistical  problems,  but 
displayed  an  unacceptably  low  Cronbach’s  alpha  of  .24  when  combined  into  a  scale.  Therefore, 
this  scale  was  not  retained,  but  both  questions  were  retained  as  individual  predictors. 

TLP.  The  first  potential  TLP  experience  scale  involved  Questions  23  through  29,  which 
asked  students  to  estimate  how  frequently  they  engaged  in  TLP  related  behaviors.  Descriptive 
analysis  of  the  questions  revealed  no  item-level  problems,  and  Cronbach’s  alpha  was  .89.  This 
scale  (henceforth  the  TLP  Frequency,  or  TLPF,  Scale)  was  retained. 

The  second  potential  TLP  experience  scale  involved  Questions  30  through  32,  which 
asked  students  frequency  questions  in  a  slightly  different  format  than  the  TLPF  scale. 

Descriptive  analysis  of  the  questions  revealed  no  item-level  problems,  and  Cronbach’s  alpha  was 
.73.  This  scale  (henceforth  the  TLP  Frequency  2,  or  TLPF2,  Scale)  was  retained. 

Prior  knowledge  tests. 

Land  navigation  prior  knowledge  test.  Descriptive  analysis  of  the  questions  revealed  no 
item-level  problems,  and  Cronbach’s  alpha  was  .58.  This  relatively  low  alpha  is  in  part  due  to 
the  nature  of  its  intended  use.  Namely,  we  deliberately  included  very  easy  and  very  hard  items. 
We  expected  that  many  people  would  be  able  to  successfully  complete  the  easy  items  but  have 
trouble  with  the  hard  ones,  thereby  reducing  internal  measures  of  reliability.  This  test  was 
retained. 

TLP  prior  knowledge  test.  Descriptive  analysis  of  the  questions  revealed  no  item-level 
problems,  and  Cronbach’s  alpha  was  .81.  More  detailed  analyses  indicated  that  the  overall  alpha 
would  not  be  improved  by  removing  any  items.  This  test  was  retained. 

Performance  criteria. 

Land  navigation.  No  item-level  analysis  was  appropriate  as  this  was  a  hands-on 
practical  exercise  and  summary  scores  only  (0  to  8  points)  were  provided.  Although  instructor 
interviews  indicated  that  the  ‘No  Go’  rate  in  previous  course  iterations  had  been  as  high  as  50%, 
the  rate  in  the  sample  was  much  lower  (13.4%). 

TLP.  Descriptive  analysis  of  the  items  revealed  no  problems,  and  the  Cronbach’s  alpha 
was  .66.  More  detailed  analyses  indicated  that  the  overall  alpha  would  not  be  improved  by 
removing  any  items. 
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Correlation  and  Regression 


The  three  demographic  variables  common  to  both  criteria  analyses  were  time  in  grade, 
time  in  service,  and  level  of  civilian  education.  There  were  six  total  variables  (five  predictors 
and  the  criterion)  correlated  in  the  land  navigation  analysis,  and  seven  in  the  TLP  analysis. 

Land  navigation.  The  predictor  variables  were  time  in  grade,  time  in  service,  civilian 
education  level,  the  LNF  Scale,  and  the  land  navigation  prior  knowledge  test  (see  Table  1.)  Now 
we  turn  to  what  we  expected  the  data  to  show.  Our  first  expectation  was  met.  Prior  knowledge 
did  significantly  predict  criterion  performance  (N=  67,  r  =  .28,/?  <  .05),  and  was  the  strongest 
(in  fact,  the  only  significant)  predictor.  As  expected  given  the  different  natures  of  the  predictor 
and  criterion,  the  correlation  was  of  small  magnitude.  However,  our  second  expectation  was  not 
met.  We  had  anticipated  that  one  or  more  of  the  experience  variables  would  be  significantly 
correlated  with  prior  knowledge,  but  not  with  the  criterion.  In  fact,  neither  prior  knowledge  nor 
the  criterion  was  predicted  by  any  of  the  experience  variables.  As  noted  above,  we  were  unable 
to  assess  our  third  expectation  as  so  few  SGI  predictions  were  supplied. 

Table  1 

Land  Navigation  Correlations 


1 

2 

3 

4 

5 

6 

7 

8 

9 

1 

— 

-.22 

-.19 

-.07 

-.01 

.17 

.12 

.18 

.28* 

2 

— 

.58* 

.03 

.12 

.06 

.01 

-.04 

-.22 

3 

— 

.12 

.04 

.17 

-.22 

-.14 

-.13 

4 

— 

-.15 

-.01 

-.27* 

-.04 

.13 

5 

— 

-.29* 

-.03 

-.32* 

.12 

6 

— 

.14 

.28* 

.24 

7 

— 

.25* 

.05 

8 

— 

.08 

Note:  *=correlations  p  <.05.  5=Question  1 

Ns  ranged  from  60  to  74.  6=Question 

l=Land  Navigation  Criterion  7=Question 

2=Time  in  grade  8=  LNF  Sea 

3=Time  in  service  9=  Land  Navigation  Prii 

4=Civilian  Education  Level 

10c 

21 

22 

ile 

or  Knowledge 
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TLP.  The  variables  in  this  analysis  were  time  in  grade,  time  in  service,  civilian 
education  level,  the  TLPF  and  TLPF2  scales,  the  TLP  prior  knowledge  test,  and  the  TLP 
criterion  (see  Table  2).  We  again  assess  our  two  expectations.  Our  first  expectation  was  met. 
Prior  knowledge  did  significantly  predict  criterion  performance  (N=74,  r  =  .40.  p  <  .01),  and  it 
was  the  strongest  (again,  the  only  significant)  predictor.  However,  our  second  expectation  was 
not  met.  We  had  expected  that  one  or  more  of  the  experience  variables  would  be  significantly 
correlated  with  the  predictor,  but  not  the  criterion.  As  with  the  land  navigation  data,  neither  prior 
knowledge  nor  the  criterion  was  predicted  by  any  of  the  experience  variables.  As  noted  above, 
we  were  unable  to  assess  our  third  expectation  as  so  few  SGI  predictions  were  supplied. 

Table  2 

TLP  Correlations 


1 

2 

3 

4 

5 

6 

7 

1 

— 

-.12 

-.02 

.18 

-.07 

-.09 

.40* 

2 

— 

.58* 

.03 

.19 

-.08 

-.06 

3 

— 

.12 

-.03 

-.12 

-.12 

4 

— 

-.02 

-.18 

.08 

5 

— 

.43* 

-.04 

6 

— 

-.10 

Note:  *=correl 
Ns  ranged  fr< 
1=TLP ( 
2=Time 
3=Time  i 

lations  p  <.05.  4=Civilian  Ed 

om  71  to  74.  5=TLP 

i'ritcrion  6=TLPI 

in  grade  7=TLP  Prioi 

n  service 

lucation  Level 

F  Scale 

72  Scale 
•  Knowledge 

The  correlation  between  prior  knowledge  and  criterion  perfonnance  found  here  is  lower 
than  that  observed  in  other  research  examining  prior  knowledge/criterion  relationships.  In  part, 
this  is  probably  due  to  the  different  nature  of  the  measures.  Other  research  tends  to  use  very 
lengthy  measures  of  prior  knowledge  consisting  of  more  than  a  hundred  items.  In  addition,  it  is 
traditional  psychometric  practice  to  generate  a  large  bank  of  test  items  and  empirically  validate 
them,  removing  the  less  predictive  ones  as  validation  proceeds.  We  did  not  have  that  luxury. 

There  is  also  another  consideration.  As  noted  above,  we  were  not  given  access  to  the 
criterion  measure  until  after  we  had  developed  the  TLP  prior  knowledge  test  and  the  TLP  exam 
had  been  administered.  We  were  therefore  unable  to  verify  the  specific  skills  being  assessed  in 
the  criterion  measure,  the  nature  of  the  criterion  questions  (e.g.,  largely  recall  vice  more 
analytically  demanding  questions),  or  the  number  of  items  drawn  from  a  specific  skill.  To  make 
this  point  clear,  in  Table  3  below  we  provide  a  crosswalk  of  the  skills  tapped  by  the  TLP  prior 
knowledge  test  and  TLP  exam. 

Table  3  illustrates  the  pitfalls  involved  in  having  to  construct  a  prior  knowledge  measure 
without  having  access  to  the  criterion.  First,  the  skills  sampled  on  the  prior  knowledge  test  and 
the  criterion  exam  do  not  overlap  as  much  as  they  could.  Consider  the  last  four  skills  listed  in 
the  table.  There  are  a  total  of  ten  items  on  the  prior  knowledge  test  which  do  tap  those  skills,  and 
none  on  the  criterion  measure  which  do  so.  Second,  the  nature  of  the  questions  differed  as  well. 
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While  the  questions  on  the  prior  knowledge  test  were  more  analytical  in  nature,  several  questions 
on  the  exam  were  more  recall  based.  Third,  even  when  skills  were  tapped  by  both  measures, 
there  were  differing  degrees  of  emphasis.  Consider  the  skill  “Conduct  Troop  Leading 
Procedures”.  There  were  four  items  on  the  prior  knowledge  test  which  tapped  that  skill,  and 
sixteen  on  the  exam. 

Table  3 

TLP  Prior  Knowledge  and  Criterion  Skill  Crosswalk 


Task 

Task  Number 

Type  Task 

Skill 

Level 

Prior  Knowledge 
Items 

TLP 

Criterion 

Use  a  map  overlay 

071-329- 

1019 

Common 

2 

14-17,  18-20 

17,  22-23, 
30 

Issue  a  Warning  Order 
(WARNO) 

071-326- 

5503 

Common 

2 

3-4 

9,  26 

Conduct  a  Leader’s 
Reconnaissance 

071-410- 

0010 

1  IB 

3 

3 

4,  9,  12,  16, 
22,  24,  27 

Prepare  an  Operation 
Overlay 

071-332- 

5000 

Common 

3 

14-20 

3,  17,  22- 
23,30 

Prepare  a  Situation  Map 

071-332- 

5021 

Common 

3 

14-20 

4-5,  17, 

22,  23,  30 

Prepare  an  oral  Operation 
Order 

071-326- 

5626 

1  IB 

4 

1,5-13 

1-5,7- 
14,  19,  24, 

26 

Integrate  threat 
capabilities  into  mission 
planning 

159-200- 

2020 

Common 

4 

18,  19 

5,6 

Conduct  Troop  Leading 
Procedures 

07-3-5036 

Collective 

Sqd/Plt 

1  -4 

1-5,7- 
10,  12-14, 
16,  19,  24, 
26,  27,  29 

Conduct  a  rehearsal 

07-3-5000 

Collective 

Sqd/Plt 

2 

9 

Prepare  for  combat 

07-3-5081 

Collective 

Sqd/Plt 

2 

9,  13 

Establish  an  observation 
post 

07-3-2018 

Collective 

Sqd/Plt 

22,  29 

Assault  a  building 

07-3-1000 

Collective 

Sqd/Plt 

14-20 

Take  action  on  contact 

07-3-1432 

Collective 

Sqd/Plt 

11 

Treat  and  evacuate 
casualties 

07-3-4045 

Collective 

Sqd/Plt 

6 

Conduct  a  passage  of  lines 
as  the  passing  unit 

07-3-1099 

Collective 

Sqd/Plt 

15 

Predicted  Versus  Observed  Performance  Categories 

Land  navigation.  As  the  total  score  correlation  did  not  meet  or  exceed  the  .37  value,  we 
proceeded  to  the  use  of  easy  and  hard  prior  knowledge  items.  First,  we  calculated  the  number  of 
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correct  responses  to  each  prior  knowledge  item  (Table  4).  We  then  calculated  perfonnance  on 
the  5  easiest  questions  (Questions  12,  4,  5,  13,  and  14)  and  examined  the  relationship  between 
that  variable  and  criterion  perfonnance.  The  resulting  display  revealed  no  interpretable  pattern. 
We  therefore  added  the  next  easiest  items  (Questions  2b  and  16)  and  repeated  the  crosstab 
procedure.  Again,  there  was  no  interpretable  pattern.  Thus,  poor  perfonnance  on  the  easiest 
questions  was  not  diagnostic  of  criterion  perfonnance.  This  may  be  due  to  the  fact  that  the 
failure  rate  for  our  sample  was  much  lower  (13.4%)  than  reported  for  earlier  iterations. 


Table  4 

Land  Navigation  Prior  Knowledge  Item  Difficulties 


Question 

Number 

%  Conect 
Responses 

Question 

Number 

(Cont.) 

%  Conect 
Responses 
(Cont.) 

12 

89.6 

2a 

44.8 

4 

83.6 

9 

41.8 

5 

76.1 

8 

35.8 

13 

73.1 

7 

23.9 

14 

71.6 

19 

19.4 

2b 

68.7 

20 

14.9 

16 

67.2 

15 

14.9 

3 

65.7 

11 

11.9 

1 

64.2 

6 

7.5 

10 

56.7 

2d 

7.5 

2c 

53.7 

18 

3 

17 

44.8 

We  therefore  turned  our  attention  to  the  hardest  prior  knowledge  items.  We  focused  on 
the  six  hardest  items  (Questions  18,  2d,  6,  11,  15,  and  20)  rather  than  five,  as  two  of  the  items 
(Questions  15  and  20)  were  equally  difficult.  We  then  tabulated  the  results.  An  interpretable 
pattern  began  to  emerge,  albeit  with  some  categorization  error.  We  therefore  added  the  next 
hardest  prior  knowledge  item  (Question  19). 

Most  individuals  who  answered  two  or  more  of  those  hard  items  conectly  scored  5  or 
more  points  on  the  land  navigation  criterion  (and  thus  achieved  a  ‘go’  status).  Increasing  the 
subset  of  items  to  the  seven  hardest  items  (Questions  18,  2d,  6,  1 1,  15,  20,  and  19)  further 
clarified  the  pattern  (see  Table  5).  We  therefore  succeeded  in  selecting  a  subset  of  prior 
knowledge  items  which  predicted  successful  criterion  performance. 

Notice  that  scoring  poorly  on  the  seven  questions  does  not  aid  in  predicting  unsuccessful 
performance  on  the  land  navigation  criterion.  In  fact,  the  majority  of  those  who  fail  the  hard 
items  (44  out  of  53)  still  successfully  completed  the  criterion  exercise.  However,  all  who  scored 
well  on  the  hard  items  (N=14)  achieved  “Go”  status.  Another  way  of  stating  this  is  that 
successful  perfonnance  on  the  hard  prior  knowledge  items  was  diagnostic,  while  unsuccessful 
performance  was  not. 
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Table  5 

Hard  Land  Navigation  Prior  Knowledge  Items  vs.  Criterion  Go/No  Go 


Land  Navigation 
Criterion 

Land  Navigation 

Prior  Knowledge  Test: 

7  Hardest  Items 

No  Go 

Go 

Row 

Totals 

0- 1  Items  Correct 

9 

44 

53 

2  or  More  Items  Correct 

0 

14 

14 

Column  Totals 

9 

58 

67 

Note.  Entries  in  cells  equal  number  of  students  who  fell  into  that  category. 


TLP.  As  the  correlation  between  the  total  predictor  and  criterion  scores  exceeded  the  .37 
threshold,  we  examined  the  ability  of  both  total  prior  knowledge  score  as  well  as  easy/hard  prior 
knowledge  items  to  predict  total  criterion  score  performance.  We  first  examined  both  the  TLP 
prior  knowledge  and  criterion  distributions  and  found  that  both  variables  could  be  broken  into 
thirds  without  unduly  distorting  the  distributions  (Table  6). 


Table  6 

TLP  Prior  Knowledge  and  Criterion  Performance  Categories  by  Thirds 


TLP  Prior  Knowledge  Scores 

TLP  Criterion  Scores 

Points 

Cumulative 
Percent  of 
Soldiers 

Performance 

Category 

(Thirds) 

Point  Range 

Cumulative 
Percent  of 
Soldiers 

Performance 

Category 

(Thirds) 

6-17 

35.1 

Bottom 

15-23 

30.6 

Bottom 

18-23 

63.5 

Middle 

24-26 

66.7 

Middle 

24-32 

100 

Top 

27-30 

100 

Top 

If  the  goal  was  to  be  as  mathematically  precise  as  possible,  we  would  be  referring 
extensively  to  predicted  criterion  scores.  But  the  goal  here  is  to  provide  end  users  with  usable 
information  derived  from  regression  procedures.  Therefore,  it  seemed  best  to  provide  range 
information  like  that  displayed  in  Table  6.  For  example,  if  there  was  a  “perfect”  relationship 
between  prior  knowledge  and  the  criterion,  then  if  an  individual  scored  between  1 8  and  23  on  the 
prior  knowledge  test,  they  would  be  expected  to  score  between  24  and  26  points  on  the  criterion. 
However,  typically  you  don’t  have  a  perfect  correlation  and  therefore  need  to  supplement  such 
information  with  a  table  that  contains  measurement  error  information  (Table  7).  Obviously, 
stronger  correlations  between  the  two  distributions  should  result  in  less  categorization  error  in 
the  resulting  table. 
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Table  7 

TLP  Prior  Knowledge  and  Criterion  Performance  Categories  by  Thirds 


Prior  Knowledge 
Scores 

Actual  Criterion  Category 

Row 

Totals 

Thirds 

Bottom 

(15-23 

pts) 

Middle 

(24-26 

pts) 

Top  (27-30 
pts) 

Bottom  (6-17  pts) 

14 

7 

5 

26 

Middle  (18-23  pts) 

4 

8 

7 

19 

Top  (24-32  pts) 

4 

11 

12 

27 

Column  Totals: 

22 

26 

24 

72 

Note.  Entries  in  cells  equal  number  of  students  who  fell  into  that  category.  Underlined  entries 
indicate  correct  classifications.  Bolded  entries  indicate  extreme  classification  errors. 

One  way  of  understanding  the  infonnation  shown  in  this  table  is  to  look  at  the  bolded 
entries,  which  represent  ‘extreme’  categorization  errors.  For  example,  of  the  26  Soldiers  who 
scored  in  the  bottom  third  of  the  prior  knowledge  distribution,  only  5  scored  in  the  top  third  of 
the  criterion  distribution.  Conversely,  of  the  27  Soldiers  who  scored  in  the  top  third  of  the  prior 
knowledge  distribution,  only  4  scored  in  the  bottom  third  of  the  criterion  distribution. 

We  next  explored  the  relationship  between  the  thirds  of  the  prior  knowledge  distribution 
and  ‘Go’  status  on  the  criterion  (Table  8).  ‘Go’  status  is  defined  by  course  personnel  as  70 
percent  or  more  items  answered  correctly.  In  this  case,  the  pattern  is  much  clearer  than  in  Table 
7.  Individuals  who  scored  in  the  top  two  thirds  of  the  prior  knowledge  test,  almost  without 
exception,  achieved  ‘Go’  status  on  the  criterion. 

Table  8 

TLP  Prior  Knowledge  Performance  Categories  and  Criterion  Go/No  Go 


Prior  Knowledge  Scores 

Criterion  Status 

Row 

Totals 

Thirds: 

No  Go 

Go 

Bottom  (6-17  pts) 

8 

18 

26 

Middle  (18-23  pts) 

1 

18 

19 

Top  (24-32  pts) 

1 

26 

27 

Column  Totals: 

10 

62 

72 

Note.  Entries  in  cells  equal  number  of  students  wl 

ho  fell  into  that  category. 

We  then  turned  to  an  analysis  of  the  relationships  between  the  easiest/hardest  prior 
knowledge  items  and  overall  TLP  criterion  score.  As  we  discussed  this  procedure  in  detail  in  the 
land  navigation  section,  we  proceed  with  minimal  comment  to  its  TLP  application.  Utilizing 
item  difficulty  information,  we  tabulated  perfonnance  on  the  five  easiest  items  (Questions  2,  3, 

9,  18a,  4d,  and  1 — see  Table  9)  against  total  criterion  performance.  However,  no  significant 
relationship  emerged,  and  subsequent  additions  of  the  next  2  easiest  questions  proved  equally 
fruitless.  We  therefore  turned  our  attention  to  the  hardest  prior  knowledge  items. 
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Table  9 

TLP  Prior  Knowledge  Item  Difficulties 


Question 

Number 

%  Correct 
Responses 

Question 

Number 

(Cont.) 

%  Correct 
Responses 
(Cont.) 

2 

91.9 

18e 

63.5 

3e 

89.2 

4b 

56.8 

9 

87.8 

4c 

56.8 

18a 

86.5 

16 

56.8 

4d 

82.4 

18f 

56.8 

1 

81.1 

10 

55.4 

17 

79.7 

4e 

54.1 

5 

78.4 

19 

51.4 

3d 

74.3 

8 

47.3 

4a 

74.3 

6 

45.9 

18g 

70.3 

20 

45.9 

15 

68.9 

7 

43.2 

14 

67.2 

3c 

40.5 

12 

66.2 

11 

33.8 

18b 

66.2 

3a 

32.4 

18c 

63.5 

13 

17.6 

18d 

63.5 

3b 

17.6 

First  we  tabulated  the  relationship  between  perfonnance  on  the  five  hardest  prior 
knowledge  items  (Questions  3b,  13,  3a,  11,  and  3c)  and  total  criterion  perfonnance.  It  was 
evident  that  a  predictive  relationship  had  emerged.  Correctly  answering  four  or  more  out  of  the 
live  hardest  items  correlated  with  scoring  between  26  and  30  points  on  the  TLP  criterion.  We 
therefore  divided  the  criterion  distribution  into  0-25  and  26-30  points  and  the  hard  items  into  0-3 
conect  and  4  or  more  correct  (see  Table  10).  We  also  mapped  the  0-3  versus  4  or  more  correct 
onto  ‘Go’  criterion  status  (see  Table  1 1).  Those  individuals  (N=  5)  who  got  4  or  more  of  the  hard 
items  correct  achieved  ‘Go’  criterion  status.  Once  again,  successful  perfonnance  on  the  hard 
items  was  more  diagnostic  than  failure  on  those  items. 

Table  10 

Hard  TLP  Prior  Knowledge  Items  vs.  Criterion  Performance  Categories 


TLP  Criterion 

TLP 

Prior  Knowledge  Test: 

5  Hardest  Items 

0-25  points 

26-30  points 

Row 

Totals 

0-3  Items  Correct 

38 

29 

67 

4  or  More  Items  Conect 

0 

5 

5 

Column  Totals 

38 

34 

72 

Note.  Entries  in  cells  equal  number  of  students  who  fell  into  that  category. 
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Table  1 1 

Hard  TLP  Prior  Knowledge  Items  vs.  Criterion  Go  /No  Go 


TLP  Criterion 

TLP 

Prior  Knowledge  Test: 

5  Hardest  Items 

No  Go 

Go 

Row 

Totals 

0-3  Items  Correct 

10 

57 

67 

4  or  More  Items  Correct 

0 

5 

5 

Column  Totals 

10 

62 

72 

Note.  Entries  in  cells  equal  number  of  students  who  fell  into  that  category 


Discussion 


Before  proceeding  to  our  recommendations  for  constructing  criterion  performance 
predictors,  we  judged  it  helpful  to  summarize  our  findings  by  repeating  how  our  expectations  of 
the  data  conformed  to  reality.  For  both  criteria,  our  first  expectation  was  supported.  Prior 
knowledge  significantly  predicted  criterion  perfonnance.  For  both  criteria,  our  second 
expectation  was  not  supported.  We  expected  that  one  or  more  of  the  experience  variables  would 
significantly  predict  prior  knowledge  but  not  criterion  performance.  In  fact,  none  of  the 
experience  variables  predicted  either  prior  knowledge  or  criterion  performance.  Our  third 
expectation  was  that  the  SGI  ratings  would  significantly  predict  prior  knowledge  but  not 
criterion  performance.  However,  despite  initial  confidence  in  their  ability  to  predict  Soldier 
criterion  performance,  instructors  often  declined  to  quantify  their  predictive  assessments. 

It  is  interesting  that  the  experience  variables  which  were  arguably  the  least  related  to 
prior  knowledge  and  criterion  perfonnance  were  time  in  grade  and  time  in  service,  as  these  are 
the  closest  analogues  to  how  experience  is  measured  in  the  research  literature.  Plausibly,  this 
underscores  our  point  that  Soldiers  fill  many  duties  and  thus  these  two  time  variables  may  reflect 
experience  in  conducting  many  different  tasks,  not  just  ones  related  to  the  course  criteria. 

We  were  surprised,  however,  that  there  the  experience  scales  did  not  significantly  predict 
prior  knowledge,  despite  being  intercorrelated  with  each  other.  One  possible  explanation  for  this 
lack  of  relationships  is  as  follows.  The  experience  scales  asked  individuals  to  reflect  on  how 
frequently  they  had  engaged  in  either  land  navigation  or  TLP  activities  during  intense  periods  of 
activity.  It  did  not  ask  individuals  to  identify  how  long  ago  those  intense  periods  of  activity 
occurred.  Therefore,  it  could  either  have  occurred  so  long  ago  that  the  frequency  judgments  are 
marred  by  misremembering  or,  alternatively,  the  frequency  judgments  are  largely  correct  but  the 
knowledge  gained  from  those  activities  has  simply  faded  in  the  intervening  time  period.  While 
these  suppositions  seem  plausible  they  must,  of  course,  remain  speculative  given  the  correlative 
nature  of  our  data. 


In  sum,  the  relationships  between  experience,  prior  knowledge,  and  criterion  performance 
from  other  research  (Bonnan,  White,  Pulakos,  &  Oppler,  1991;  Borman,  White,  &  Dorsey,  1995; 
Palumbo,  Miller,  Shalin,  &  Steel- Johnson,  2005;  Schmidt  &  Hunter,  1993;  Schmidt,  Hunter,  & 
Outerbridge,  1986)  were  only  partially  replicated  here.  Prior  knowledge  was  indeed  the 
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strongest  (in  fact,  the  only)  predictor  of  criterion  perfonnance.  However,  the  failure  of  the 
experience  variables  to  predict  prior  knowledge  underscores  some  of  the  differences  between  the 
contexts  of  Army  institutional  training  and  research  settings. 

Also  relatively  consistent  across  the  two  analyses  were  the  results  of  using  easy  and  hard 
prior  knowledge  items  to  predict  criterion  performance.  For  both  of  the  criteria,  predicting  poor 
performance  using  easy  prior  knowledge  items  was  unsuccessful.  Conversely,  using  hard  prior 
knowledge  items  resulted  in  identifying  a  small  set  of  Soldiers  who  almost  without  fail 
performed  successfully  on  the  criteria.  Again,  we  note  that  it  was  successful  performance  on  the 
hard  prior  knowledge  items  which  was  diagnostic.  Predicting  poor  criterion  performance 
required  using  the  total  score  from  the  TLP  prior  knowledge  test. 

Recommendations 

In  the  absence  of  replication,  our  findings  should  be  accepted  provisionally.  However, 
the  fact  that  the  findings  were  consistent  across  the  two  analyses  does  lend  credence  to  our 
results.  We  are  therefore  confident  in  the  following  recommendations  based  upon  these  data. 

Use  Prior  Knowledge  as  a  Predictor 

When  possible,  using  prior  knowledge  as  a  predictor  is  a  good  bet.  As  discussed  in  the 
introduction  of  this  paper,  prior  knowledge  captures  the  joint  effects  of  both  mental  ability  and 
experience  within  a  domain.  This  was  borne  out  by  the  fact  that  in  both  of  our  analyses,  prior 
knowledge  alone  significantly  predicted  perfonnance. 

Combine  Prior  Knowledge  With  Narrowly  Focused  Demographic  Variables 

The  general  sorts  of  demographic  variables  which  anecdotal  evidence  suggests  instructors 
use  to  assess  current  and  future  performance  were  not  predictive.  However,  it  seems  to  us  that 
using  demographic  variables  to  ferret  out  subgroup  differences  is  a  promising  avenue.  Although 
the  instructors  indicated  no  awareness  of  broad  subgroup  differences  in  the  ALC  population,  our 
prior  (Schaefer,  Bencaz,  Bush,  &  Price,  2010)  and  current  (Schaefer,  Blankenbeckler,  & 

Lipinski,  2011)  research  has  found  at  least  two  Army  courses  in  which  subgroup  differences  (as 
measured  by  demographic  variables)  exhibited  starkly  different  predictor-criterion  relationships. 
Encouragingly,  in  both  of  those  courses  the  subgroup  populations  were  brought  to  our  attention 
by  the  course  instructors,  indicating  that  such  differences  are  sometimes  known  to  course 
personnel. 

Estimate  Total  Score  and  Easy/Hard  Item  Relationships  When  Validating  Predictors 

If  the  correlation  between  prior  knowledge  and  criterion  total  score  is  large  enough  (using 
our  given  rule,  .37  or  more)  then  crosstabs  can  be  used  to  generate  information  usable  by  course 
personnel.  Such  information  can  then  be  leveraged  to  probabilistically  categorize  future  criterion 
performance  throughout  the  entire  examined  criterion  range.  Further  information  can  be  gleaned 
by  examining  the  ability  of  hard  (and,  in  theory,  easy)  prior  knowledge  items  to  predict  criterion 
performance. 
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When  no  large  (.37  or  greater)  total  prior  knowledge-criterion  score  correlation  is  present, 
it  is  still  possible  to  use  sets  of  hard  items  to  predict  who  will  do  extremely  well.  However,  as 
the  TLP  data  demonstrated,  even  when  a  strong  correlation  is  present,  using  subsets  of  hard  prior 
knowledge  items  can  be  helpful.  We  are  less  sure  how  feasible  it  is  to  use  sets  of  easy  items  to 
predict  who  will  not  do  very  well.  This  is  probably  a  function  of  the  different  difficulty  levels  of 
the  prior  knowledge  test,  the  criterion  test,  and  a  variety  of  other  factors.  In  any  case,  we  were 
unable  to  accomplish  this  in  the  current  effort. 

It  is  probably  also  prudent  to  administer  the  complete  exam  rather  than  just  the  subsets  of 
easy  and  hard  items.  In  the  case  of  the  TLP  prior  knowledge  test,  for  example,  some  of  the 
questions  build  upon  earlier  ones  and  are  placed  in  a  given  context.  Having  students  answer  just 
some  of  the  questions  would  require  revising  the  test,  and  thus  calling  into  question  all  of  the 
psychometric  test  information  we  reported  here.  In  the  case  of  the  land  navigation  exercise, 
administering  the  hard  items  alone  would  likely  demotivate  individuals  who  were  daunted  by  the 
skill  level  required.  Having  those  individuals  complete  a  prior  set  of  questions  which  they  could 
answer  might  counteract  this.  This  does  not  require,  of  course,  that  the  instructor  grade  the  entire 
measure.  Rather,  the  instructor  could  administer  the  entire  instrument  and  focus  on  only  the  hard 
items  when  grading. 

Explore  the  Predictor-Criterion  Relationship  in  Multiple  Ways 

The  previous  paragraph  dovetails  with  the  following  contention:  just  because  one  metric 
(e.g.,  a  correlation)  indicates  a  weak  relationship  between  a  predictor  and  a  criterion  does  not 
mean  that  another  method  (here,  hard  prior  knowledge  items)  will  not  reveal  a  more  useful 
relationship.  In  addition,  exploring  the  relationships  between  performance  on  hard  predictor 
items  and  overall  criterion  performance  was  very  illuminating.  It  is  perfectly  possible  that 
viewing  the  data  through  a  specific  lens-say,  how  hard  item  performance  relates  to  ‘Go’  status  on 
a  criterion — can  fail  to  reveal  a  relationship  which  would  be  readily  seen  if  explored  in  other 
ways. 


Greater  efficiency  appears  to  be  obtained  when  the  design  of  predictor  items  is  focused 
on  the  actual  criterion.  While  predictor  items  focused  on  specific  tasks,  knowledge,  or  skills 
have  some  value,  that  value  seems  to  increase  when  assessment  methods  parallel  or  coincide 
with  assessment  techniques  used  in  the  criterion.  When  this  paralleling  of  structures  is  not 
possible  the  questions  should  be  designed  to  not  merely  focus  on  simple  recall  or  the  ability  to 
list  facts  and  terms,  but  should  require  the  use  of  information  and  application  of  principles  in  the 
correct  manner.  This  design  provides  improved  insights  to  differences  in  conceptual 
understanding  not  brought  out  by  simple  recall. 

When  Possible,  Use  Hands-on  Predictors  with  Hands-on  Criteria 

In  this  effort,  we  judged  the  potential  savings  in  time  gained  by  using  a  paper  and  pencil 
predictor  for  a  hands-on  practical  land  navigation  exercise  worth  the  effort.  However,  not  all 
hands-on  criteria  require  such  investment  in  time  and  effort.  For  example,  if  the  hands-on 
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criterion  can  be  performed  on  readily  available  equipment,  it  is  sensible  to  have  Soldiers 
demonstrate  directly  on  the  equipment  rather  than  rely  on  a  paper  and  pencil  measure. 

Develop  a  Big  Set  of  Predictor  Test  Items  for  Initial  Exploration 

We  did  not  have  the  luxury  of  developing  a  big  set  of  predictor  items,  although  this  is 
standard  practice  in  large  scale  psychometric  efforts.  This  allows  for  greater  flexibility  in 
including  items  which  range  from  very  easy  to  very  hard.  In  turn,  this  would  perhaps  allow  for 
more  refined  use  of  easy  and  hard  items. 

Focus  on  Narrow  Criteria 

In  our  prior  research  (Schaefer,  Bencaz,  Bush,  &  Price,  2010),  we  focused  on  broad 
psychological  traits  (e.g.,  metacognition)  and  broad  measures  of  achievement  (e.g.,  class 
average).  However,  given  the  relative  success  of  using  prior  knowledge  measures  as  predictors, 
a  different  tact  is  advisable. 

Constructing  prior  knowledge  measures  which  would  tap  the  content  of  an  entire  course 
seems  ill-advised.  First,  developing  and  administering  such  a  measure  would  take  an  inordinate 
amount  of  time.  Second,  it  is  unclear  how  helpful  such  information  would  be.  If  the  person  does 
poorly  on  all  aspects  of  the  measure,  do  you  have  to  tailor  the  entire  course  around  them?  Third, 
such  an  approach  does  not  lend  itself  to  assessment  throughout  a  course. 

It  seems  more  feasible  to  make  ‘mini-tests’  available  to  instructors  prior  to  blocks  of 
instruction  or  training  of  tasks  that  are  important  in  terms  of  money,  core  objectives,  establishing 
foundational  knowledge  and  skills,  or  just  plain  difficult.  Then  decisions  can  be  made  regarding 
what  kind  of  tailoring  (if  any)  is  warranted  on  that  particular  block  of  training. 

In  sum,  making  intelligent  tailored  training  decisions  will  require  a  unique  blend  of 
testing  and  subject  matter  expertise.  The  need  for  testing  expertise  is  obvious,  requiring 
knowledge  of  test  construction  and  validation  procedures.  However,  the  need  for  subject  matter 
expertise  is  at  least  as  (if  not  more)  important.  Subject  matter  experts  will  be  required  to  help 
test  creators  detennine  suitable  items  for  tapping  prerequisite  skills  and  experiences.  In  addition, 
subject  matter  experts  can  help  test  creators  determine  what  kinds  of  demographics  should  be 
included  to  test  for  subpopulation  differences.  The  two  subpopulation  differences  found  in  our 
prior  (Schaefer,  Bencaz,  Bush,  &  Price,  2010)  and  current  (Schaefer,  Blankenbeckler,  & 

Lipinski,  2011)  research  were  brought  to  our  attention  by  course  personnel  prior  to  test 
construction.  Developing  research  teams  with  the  appropriate  psychometric  and  military 
expertise  will  require  careful  investment  of  resources,  further  suggesting  the  need  for  targeting 
areas  in  which  tailoring  will  yield  the  most  benefit. 
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Appendix  A 

Course  Selection  Criteria 


1 .  Number  of  students  in  each  course  -  Each  student  arrives  at  a  course  with  his/her  own  KSE 

(knowledge,  skills  experiences),  gained  over  years.  Therefore,  theoretically,  the  larger  the 
number  of  students  in  a  course,  the  greater  the  “potential”  for  differences  in  KSE. 
However,  keep  in  mind  that  even  though  there  might  be  a  large  number  of  students,  it’s 
possible  that  a  majority  will  have  similar  KSE,  with  only  some  minority  having  different 
KSE.  Ensure  that  selected  courses  have  a  large  enough  “sample  size”  of  students  with 
differing  KSE. 

Guideline:  Courses  with  a  larger  number  of  students  are  more  likely  to  have  more  differences  in 
KSE. 

2.  Multiple  MOSs  -  Each  MOS  (and  branch/specialty  for  officers)  of  the  Army  has  some  unique 

training  requirements,  skills  and  tasks.  Therefore,  personnel  from  varied  MOSs 
(branches/specialties)  will  arrive  at  a  course  with  differing  KSE.  However,  keep  in  mind 
that  even  though  there  might  be  a  large  number  MOSs  (branches/specialties),  it’s  possible 
that  a  majority  of  students  will  have  a  common  MOS  (branch/specialty),  with  only  some 
minority  being  a  different  MOS  (branch/specialty).  Ensure  that  selected  courses  have  a 
large  enough  “sample  size”  of  students  with  different  MOS  (branch/specialty). 

Guideline:  The  larger  the  variety  of  MOSs  (and  branch/specialty  for  officers)  attending  the 

course,  the  greater  the  likelihood  of  differences  in  KSE.  Also  consider  that  some  MOSs 
(branch/specialties)  are  so  different  that  those  attending  a  course  will  increase  the 
likelihood  of  different  KSE.  (Example:  Soldiers  from  infantry,  armor  and  even  engineer 
areas  are  much  more  similar  in  many  aspects  of  KSE  than  Soldiers  from  chaplain 
assistant  or  transportation  areas.)  An  ideal  situation  would  be  a  course  with  2-3  well- 
represented,  qualitatively  different  MOSs. 

3.  Course  Length  (topic/subiect)  -  With  the  exception  of  Initial  Entry  Training  (IET)  courses, 

longer  courses  (more  than  45  days)  are  generally  for  NCO  and  officer  professional 
development  and  are  not  usually  focused  on  a  specific  skill  or  capability.  As  the  level  of 
the  course  increases  (e.g.  from  ALC  [E-6]  to  SLC  [E-7]  or  from  Officer  Basic  Courses 
[0-1]  to  Captains’  Career  Courses  [0-3])  the  military  KSE  will  likely  increase. 

Personnel  attending  the  higher  level  courses  will  have  had  more  time  in  Service  and  more 
assignments.  However,  the  overall  general,  military  experience  will  become  more 
common  as  the  time  in  Service  increases.  Keep  in  mind  that  the  focus  is  on  the  technical 
skill  areas  (not  soft  skills)  which  will  only  be  a  portion  of  the  course. 

Guideline:  Generally,  the  shorter  courses  that  are  not  designed  for  a  specific  MOS/branch  are 
more  likely  to  have  differences  in  more  general  KSE,  while  the  longer  professional 
development  courses  will  have  greater  differences  in  specific  military  assignment  KSE 
areas.  Consider  only  technical  portions  of  professional  development  courses. 
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4.  Course  Content  -  The  nature  of  the  course  content  (“soft  skills”  versus  technical  skills)  will 

have  implications  for  how  easily  prior  knowledge  can  be  measured  or  how  easily 
performance  can  be  measured.  Generally,  need  to  consider  the  technical  task  areas  for 
courses  where  prior  knowledge  can  be  measured  and  avoid  attempts  to  measure  “soft 
skill”  areas.  Consider  blocks  of  training  within  courses  rather  than  an  entire  course, 
especially  if  the  block  of  training  is  a  critical  technical  skill  area.  Also,  students  are  more 
likely  to  possess  differences  in  KSE  in  the  more  technical  areas  than  in  the  “soft  skill” 
areas. 

Guideline:  Differences  in  KSE  will  generally  be  more  important  in  courses  and  blocks  of  training 
with  structured,  sequential  technical  skill  areas  that  are  critical  for  course  completion. 
Unstructured  and  non-sequential  courses  and  blocks  of  training  will  generally  involve 
more  “soft  skill”  areas  and  the  differences  in  KSE  will  have  less  impact. 

5.  Prerequisites  -  Students  attending  higher  level  courses  (e.g.,  Sergeant  Major  Academy  as 

opposed  to  SLC  or  ALC)  will  generally  begin  the  course  with  a  more  common  skill  level 
in  the  area  to  be  trained  in  the  course.  If  course  prerequisites  are  established  and 
enforced,  the  likelihood  of  prior  KSE  that  could  impact  the  course  training  may  be 
minimal. 

Guideline:  “Basic”  and  “intermediate”  level  courses  are  more  likely  than  more  “advanced”  level 
courses  to  have  students  with  differences  in  KSE  that  matter. 

6.  Mandatory  course  completion  -  Courses  that  must  be  successfully  completed  to  continue 

Service  within  the  military  (e.g.,  professional  development  courses  versus  basic  digital 
skills)  are  more  likely  to  have  students  attending  with  greater  differences  in  KSE.  The 
intent  of  the  courses  is  generally  to  allow  students  to  “cross-level”  the  military 
experiences  they  have  gained  so  all  can  move  forward  with  a  more  common  and 
complete  understanding  of  the  military. 

Guideline:  Mandatory  professional  development  courses  are  more  likely  to  have  measurable 
differences  in  KSE  than  more  general  subject  area  courses.  Consider  only  technical 
portions  of  professional  development  courses,  not  the  general  “soft  skills”. 

7.  Volunteer  or  selected  for  course  -  Generally,  courses  with  attendees  who  must  volunteer  (e.g., 

Airborne)  are  generally  people  who  perceive  a  beneficial  outcome  from  the  completion  of 
the  course,  either  personal  gratitude  or  professional  enhancement.  Personnel  who  are 
selected  for  course  attendance  based  on  some  criteria  (e.g.,  Drill  Sergeant)  may  not  have 
the  same  perceptions  or  motivation.  Selection  criteria  will  usually  consider  identifiable 
areas  of  KSE.  Therefore,  it  could  be  presumed  that  courses  with  all  volunteers  are  more 
likely  to  have  a  greater  difference  in  KSE  than  courses  with  central  selection  processes. 

Guideline:  Courses  that  have  both  volunteers  and  selectees  have  a  high  possibility  of  extreme 
differences  of  KSE,  as  well  as  all  volunteer  courses. 
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Other  Considerations 


1 .  Number  of  courses  that  can  be  affected  -  Once  potential  courses  for  differences  in 
KSE  have  been  identified,  one  of  the  down-select  factors  should  consider  the  number  of  similar 
courses  taught  at  multiple  locations  who  could  benefit  from  the  results  of  this  investigation;  to 
provide  the  Army  a  “bigger  bang  for  the  buck.” 

2.  Decisions  as  to  which  courses  to  examine  for  this  project  can  be  based  on  the 
established  criteria.  In  this  decision  process,  interactions  between/among  criteria  should  also  be 
considered  as  an  important  factor.  Since  only  5  courses  will  be  selected  to  visit  to  gather 
information  on  potential  KSE  to  measure,  a  further  consideration  is  the  number  of  potential 
courses  at  an  installation  that  offer  potential.  That  is,  if  multiple  courses  offer  the  same  potential 
for  measuring  KSE,  priority  should  be  given  to  multiple  courses  at  the  same  installation  in  order 
to  maximize  benefit  of  travel. 

At  the  end  of  this  criteria  definition  process,  we  will  compile  the  assessment  for  each 
criterion  for  10  courses  (some  information  will  come  from  web  sites  and  other  from  telephone 
calls).  When  pertinent  information  is  available  we  will  establish  a  relatively  simple  check  list  to 
apply  to  the  courses  (see  below).  Keep  in  mind,  our  purpose  in  this  exercise  is  to  identify  the  5 
courses  we  would  like  to  visit  to  help  determine  which  KSE  and  what  measures  would  be  most 
appropriate.  Something  like  the  following  rating  scale  might  work. 

Use  a  rating  scale: 


0  3  5 

(Very  slim  chance  of  differing  KSE)  (Almost  certain  of  differing  KSE) 


Selection  Criteria 

Course 

1 

2 

3 

4 

5 

6 

7 

A-3 


Appendix  B 

Small  Group  Instructor  Predictions 


The  purpose  of  this  form  is  to  gain  insight  to  your  intuition  and  observations  in  assessing  student 
knowledge,  skills,  and  experiences.  Many  trainers  have  indicated  that  they  are  able  to  assess 
student  potential  and  performance  in  general  and/or  for  specific  subjects  and  skills  early  in  the 
course.  Please  rate  the  students  in  your  instructional  group  and  any  other  students  in  the  course 
that  your  intuition,  observations,  or  impressions  have  caused  you  to  assess.  Place  an  X  or  S  in 
the  appropriate  box  for  Land  Navigation  and  TLP/Tactical  Operations. 


Student 

Roster 

Number 

Assessment  of  the  Students  Future  Academic  Performance 

Land  Navigation 

1 

fLP 

Top 

25% 

Middle 

50% 

Lower 

25% 

Cannot 

Evaluate 

Top  25% 

Middle 

50% 

Lower  25% 

Cannot 

Evaluate 

Students  in  My  Group 

Other  Students  in 
the  Course 
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Demographic  Questionnaire  and  Experience  Scales 

L  BIOGRAPHICAL  INFORMATION 
1.  Roster  Number 


2.  Rank  (circle  one) 

SGT  SGT(P)  SSG  SSG  (P)  SFC  Other: 


TIG/TIS 

Years 

Months 

3.  Time  in  Grade 

4.  Time  in  Service 

5.  Primary  MOS  (circle  one) 

1 IB  11C 

5.  A.  Do  you  have  prior 
experience  in  another  Service 
and/or  MOS? 

YES  NO  (If  YES,  see  5.B) 

5.B.  Other  MOS  &  Service: 

.  Service/Branch: 

MOS: 

6.  Service  Status  (circle  one) 

Active  Duty  National  Guard  Army  Reserve 


7.  Year  completed  WLC  or  PNCOC: 

8.  Civilian  Education  Level  (circle  highest  level  of  education) 

NonHSG  GED  HS  Diploma  Some  College  (no  degree) 

Associates  Degree  Bachelors  Degree  Graduate  Work  Master’s  Degree 

8.  A.  If  undergraduate  or  graduate  degree  state  type: 
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9.  Deployment  History  (Most  recent  first) 

Return  Y ear 

Iraq 

Afghan 

Other 

Duty  Position 

Unit  Primary  Mission 

e.g.  2007 

X 

Fire  Team  Leader 

Route  Security 

(Continue  if  more  dep] 

loyment  experience) 

10.  Experience  with  digital  C3I  systems  - 

Navigation  Support 

10.A.  Have  you  used  global  positioning 
systems  (GPS)  or  other  navigation  aids  to 

Circle  one: 

include  FBCB2,  LandWarrior,  etc.? 

YES 

NO 

10.B.  If  YES,  did  you  also  have  available 
and  use  paper  maps  and  a  lensatic  compass 

Circle  one: 

to  assist ,  follow  movements,  etc.? 

YES 

NO 

10.C.  If  YES,  were  maps  and  a  lensatic 
compass  your  primary  navigation  aids? 

Circle  one: 

YES 

NO 

My  duties  did  not  require  navigation. 

11.  Experience  with  digital  C3I  systems  - 

Operations  and  Situational  Awareness  Support 

Have  you  used  FBCB2,  LandWarrior,  or 

Circle  one: 

another  digital  command  and  control  or 

situational  awareness  support  system? 

YES 

NO 
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PART  II.  SKILLS  ASSESSMENT 


This  section  will  ask  you  to  self-assess  selected  skills. 

A.  There  are  a  number  of  specific  skills  that  impact  your  ability  to  navigate  using  a  map  and  compass. 
Consider  deployments,  training,  and  periods  in  your  career  when  you  have  been  required  to  use  your 
navigation  skills  the  most.  Use  the  scale  provided  to  rate  your  frequency  of  use  for  the  tasks  indicated. 
Place  check  marks  in  the  appropriate  cell  for  each  task  or  skill. 


Task  or  Skill 

Frequency  of  Use  Scale 

Never 

Rarely 

Monthly 

Weekly 

Daily 

12.  Measure  distance  travelled 
accurately  using  pace  count  and  dead 
reckoning  techniques. 

9.5% 

62.2% 

18.9% 

2.7% 

6.8% 

13.  Recognizing  and  associate 
terrain  features  on  the  map  and  on  the 
ground  around  you. 

4.0% 

32.4% 

27.0% 

10.8% 

25.7% 

14.  Use  terrain  association  and/or 
azimuths  to  known  points  to 
determine  your  location. 

5.4% 

49.0% 

20.3% 

8.1% 

17.6% 

15.  Given  a  grid  coordinate,  finding 
a  point  or  a  natural  or  man-made 
feature  on  a  map. 

2.7% 

31.1% 

22.9% 

18.9% 

24.3% 

16.  Given  a  point  or 
feature  on  a  map  and  a 
protractor  with  grid  scale, 
determine  the  grid 
coordinate  to  the  nearest: 

±  100  m 

9.5% 

43.2% 

20.2% 

9.5% 

16.2% 

±  10m 

9.5% 

40.5% 

24.3% 

10.8% 

13.5% 

±  1  m 

21.6% 

54.0% 

13.5% 

4.1% 

5.4% 

17.  Measure  distance  between  two 
points  on  a  map. 

2.7% 

29.7% 

31.1% 

20.3% 

16.2% 

18.  Convert  magnetic  azimuths  to 
grid  azimuths  and  grid  azimuths  to 
magnetic  azimuths. 

9.5% 

54.1% 

22.9% 

81.1% 

5.4% 

19.  Navigate  around  obstacles  or 
dangers,  while  maintaining  location 
awareness  and  recover  to  your 
intended  route. 

4.1% 

33.8% 

29.7% 

13.5% 

18.9% 

20.  Given  a  start  and  end  point  on  a 
map  and  an  enemy  situation,  identify 
the  best  route. 

6.8% 

31.1% 

17.6% 

20.3% 

24.3% 
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21 .  Your  new  platoon  leader  is  planning  his  first  mission,  a  dismounted  movement  to  contact  in  hilly, 
forested  terrain,  similar  to  Fort  Benning,  GA.  Fie  asks  for  your  assistance.  Fie  needs  someone  skilled 
with  a  map  and  compass;  the  platoon’s  global  positioning  systems  are  not  operational.  Which  statement 
below  best  describes  your  land  navigation  skills. 

l.Most  NCOs 
in  the  platoon 
can  navigate 
better  than  I 

can. 

2.1  am  capable 
of  moving  on  a 
designated 
azimuth  within 
±  3  degrees  for 
200  meters  or 
farther. 

3.1  am  capable 
of  moving  on  a 
designated 
azimuth  within 
±  3  degrees  for 

2  kilometers  or 
farther. 

4.1  am  capable 
of  guiding  my 
unit  to  within 
±100  meters 
of  designated 
check  points  or 
grid  locations. 

5. Capable  of 
guiding  my 
unit  to  within 

1 0  meters  of 
designated 
check  points. 

6. Confident  to 
call  for  and 
adjust  “danger 
close”  indirect 
fires. 

2.7% 

6.8% 

41.9% 

37.8% 

10.8% 

22.  Your  company  is  conducting  a  cordon  and  search  mission.  The  objective  is  a  small  village,  a 
settlement  of  30  masonry  and  mud  dwellings  in  a  deep  river  valley.  The  valley  floor  is  800  to  1000 
meters  across  divided  by  the  Black  River,  with  widely  separated  foot  fords.  Vegetation  in  the  valley  floor 
is  chest-high  prairie  grasses  and  thick  groves  of  cedar  trees,  dotted  with  farm  plots  100  X  100  meters. 

Your  platoon’s  mission  is  the  cordon.  Based  on  your  navigation  skills  alone,  which  mission  are  vou  best 
suited  to  lead?  Circle  one  response. 

l.Drop  off  the  rear 
of  the  convoy 
going  in  and 
establish  a  TCP  on 
the  main  road. 

2.  Move  beyond 
the  village  and 
interdict  foot 
traffic  using  the 
shallow  ford 
across  the  Black 
River. 

3. Move  to 
observation  point 
on  Hill  763,  be 
prepared  to  move 
to  interdict  traffic 
along  the  trail 
networks  at  the 
base  of  the  Hill. 

4.Move  by  a  cross 
country  route  to  a 
specified  choke 
point  on  the  foot 
path  out  of  the 
mountains. 
Establish  a  TCP  to 
control 
dismounted 
movement. 

5. Act  as  the 
company  quick 
reaction  force.  Be 
prepared  to 
quickly  move  to 
any  point  to 
reinforce  another 
element  or  assume 
a  mission. 

2.7% 

8.1% 

25.7% 

36.5% 

27.0% 
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B.  There  are  a  number  of  specific  skills  that  impact  your  ability  to  conduct  Troop  Leading  Procedure 
(TLP),  receive,  understand,  and  issue  operations  orders.  Use  the  scale  provided  to  rate  your  frequency  of 
use  for  each  task  indicated.  Consider  your  frequency  of  use  during  a  recent  deployments  or  a  period  of 
intense  training.  Place  check  marks  in  the  appropriate  cell  for  each  task  or  skill. 


Task  or  Skill 

Frequency  of  Use  Scale 

Never 

Rarely 

Monthly 

Weekly 

Daily 

23.  Analyze  specified  and  implied 
tasks  assigned  in  the  platoon  order  to 
determine  the  tasks  and  purposes  for 
fire  teams. 

2.7% 

16.2% 

21.6% 

29.7% 

29.7% 

24.  Analyze  terrain  to  determine 
routes,  position  elements,  or  site 
weapons. 

1.4% 

14.9% 

22.9% 

17.6% 

43.2% 

25.  Recommend  or  refine  targets  to 
support  the  scheme  of  maneuver  or 
defensive  positions. 

9.5% 

25.7% 

22.9% 

20.2% 

21.6% 

26.  Issue  a  complete  order  to  my 
subordinates  and  assure  their 
understanding. 

8.1% 

18.9% 

18.9% 

21.6% 

32.4% 

27.  Conduct  pre-combat  inspections 
of  my  unit  with  a  focus  on  the  special 
needs  of  the  mission  or  the  execution 
of  key  assigned  tasks. 

0% 

4.1% 

13.5% 

14.9% 

67.6% 

28.  Rehearse  the  mission  or  critical 
tasks  of  the  mission  to  prepare  my 
unit. 

2.7% 

4.1% 

17.6% 

33.8% 

41.9% 

29.  Used  a  sand  table,  table  top, 
mission  walk  through,  or  a  part  or 
full  force  rehearsal  to  prepare  for  a 
mission. 

2.7% 

17.6% 

20.3% 

25.7% 

33.8% 
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30.  Indicate  the  experience  you  have  had  with  TLP,  specifically  receiving  orders  from  your  platoon 
leader  or  company  commander. 

1.1  seldom 
received  an  order. 

The  normal 
process  in  the  unit 
was  to  move  out 
and  FRAGO  as  the 
situation  evolved. 

2.1  sometimes 
received  an 
abbreviated  order 
providing  a 
mission  statement 
with  an  overview 
of  critical  aspects 
of  the  mission. 

3. 1  sometimes 
received  an 
abbreviated  order 
providing  a 
mission  statement 
with  task  and 
purpose  statements 
for  subordinate 
units. 

4.1  almost  always 
received  an  order 
with  the  mission, 
details  about  the 
enemy  situation, 
and  specifics  of 
the  task  and 
purpose  for  my 
unit. 

5.1  always 
received 
complete 
orders. 
Situation, 
mission, 
subunit  tasks, 
and  admin/log 
details  were 
provided. 

4.1% 

9.5% 

22.9% 

36.5% 

25.7% 

3 1 .  Indicate  the  experience  you  have  had  with  TLP,  specifically  receiving  operations  graphics  and 
graphic  enemy  situation  updates  from  your  platoon  and/or  company  headquarters. 

1.1  was  never 
provided 
operations 
graphics,  mission 
graphics,  or 
friendly  or  enemy 
situation  updates. 

2.1  have  seldom 
received 
operations 
graphics  or 
graphical  friendly 
or  enemy  situation 
updates. 

3. 1  sometimes 
received 
operations 
graphics  for  the 
mission  and 
graphical  enemy 
and  friendly 
situation  updates. 

4.1  normally 
received 
operations 
graphics  for  the 
mission  and 
graphical  enemy 
and  friendly 
situation  updates. 

5.1  always 
received  orders. 

They  were 
complete  with 
graphics  and 
issued  with 
sufficient  time  for 
subordinate  units 
to  conduct  TLP. 

4.1% 

5.4% 

17.6% 

47.3% 

25.7% 

32.  Indicate  the  experience  you  have  had  with  TLP,  specifically  in  preparing  your  unit  for  the  mission 
with  pre-combat  inspections,  rehearsals,  and  training. 

1 .1  never  had  time 
to  prepare  for  the 
next  mission. 

2. Any  mission 
preparation  was 
done  by  the  SOP. 

I  never  had 
mission  details  for 
preparation. 

3.  Seldom  did  we 
have  mission 
details.  Most 
mission 

preparations  were 
based  on  our  SOP. 

4.Frequently  we 
were  able  to  use  a 
table  top  rehearsal 
or  sand  table 
prepare  for  the 
mission.  Pre¬ 
combat 

inspections  were 
always  based  on 
mission  specifics. 

5. Often  we  did  full 
scale  rehearsals 
involving  all 
Soldiers.  We 
sometimes 
retrained/practiced 
skills  (room 
clearing,  danger 
areas,  etc.).  PCIs 
were  mission 
focused. 

1.4% 

22.9% 

45.9% 

29.7% 
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Appendix  D 

Land  Navigation  Prior  Knowledge  Test 

1 .  Military  standard  topographic  maps  use  colors  and  symbols  to  depict  features  and 
conditions.  Select  the  item  below  which  accurately  reflects  the  method  that  would  be 
used  to  depict  a  spot  elevation.  (Circle  one.) 

[Question  1  =  64.2%  correct] 
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2.  Military  topographic  maps  use  colors  and  symbols  to  depict  various  types  of  ground 
cover,  forest  types,  and  cultivation.  Match  the  symbol  with  the  type  of  plant  life  or 
vegetation  that  it  depicts.  Enter  the  letter  for  the  symbol  in  the  appropriate  blank. 

(NOTE:  Not  all  letters  may  match  descriptions;  if  no  match  is  found  place  an  “X”  in  the 
answer  space.) 

[Question  2-a  =  44.8%,  Q2-b  =  68.7%,  Q2-c  =  53.7%,  Q2-d  =  7.5%  correct] 

_ _ Brush  and  scrub _ Orchards _ Rice  paddies _ Open  fields 
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3.  The  contour  interval  of  10  feet  is  identified  in  the  legend  of  for  this  map.  Which 
labeled  arrow  contains  the  steepest  climb?  (Circle  the  best  answer.) 


[Question  3  =  65.7% 
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4.  From  your  location  at  the  center  of  the  triangle  in  the  dashed  orange  box,  what  is  the 
direction  of  movement  to  the  road  junction  in  the  black  box?  (Circle  the  correct  answer.) 

[Question  4  =  83.6%  correct] 


5.  You  remain  at  the  same  location  as  in  the  previous  question.  What  is  the 
approximate  grid  azimuth  to  the  road  junction  in  the  black  box?  (Circle  the  correct 
answer.) 

[Question  5  =  76.1%  correct] 

315°  225°  25°  157°  190°  362°  265° 
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6.  You  have  picked  one  of  your  injured  Soldiers.  He  has  been  released  from  the 
hospital  and  returned  to  duty.  You  are  being  flown  back  to  an  LZ  near  your  unit’s 
current  location.  A  message  from  your  platoon  sergeant  provided  a  link  up  grid  location 
of  52S  BG  9965  5998.  Before  departing  the  hospital  you  need  to  draw  a  set  of  maps  of 
your  area  of  operations.  What  element  of  location  information  is  provided  by  the  letters 
“BG”?  (Circle  the  best  answer.) 

[Question  6  =  7.5%  correct] 

A.  Grid  zone  designation 

B.  Map  sheet  number 

C.  Grid  coordinates 

D.  100,000  meter  grid  square  identification 

E.  None  of  the  above  is  correct. 
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7.  Your  mounted  patrol  is  moving  to  the  Southeast;  your  lead  element  is  located  at  16S 
FA  99342  90612.  Reports  from  a  supporting  UAV  element  indicate  that  enemy  infantry 
are  in  the  clearing  at  16S  GA  00202  90375.  Approximately  (within  ±  10  meters)  how  far 
is  your  patrol  from  the  hostile  force?  (Circle  the  correct  answer.) 


[Ques 


ion  7  =  23.9%  correct] 
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A.  1,000  meters  D.  750  meters 

B.  900  meters  E.  700  meters 

C.  800  meters  F.  None  of  the  previous  answers 

are  correct. 

8.  Your  unit  will  be  conducting  “knock  and  search”  missions  in  a  neighborhood  of  a 
major  city.  The  planned  area  of  operations  measures  approximately  6  city  blocks  by  8 
city  blocks  (1 .2  kilometers  X  1 .6  kilometers).  The  Platoon  Sergeant  indicates  that  new 
maps  are  available  in  varied  scales.  Which  scale  of  map  would  provide  the  greatest 
terrain  details  for  planning  and  battle  tracking?  (Circle  the  correct  answer.) 

[Question  8  =  35.8%  correct] 


A.  1:250,000 

D.  1:25,000 

B.  1:100,000 

E.  1:12,500 

C.  1:50,000 
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9.  Maps  can  be  supplemented  by  overhead  imagery  to  identify  changes  to  both 
manmade  and  natural  features.  Current  photos  help  determine  when  new  roads  or 
structures  have  been  added,  unused  trails  disappear,  and  patterns  of  vegetation 
change.  Generally,  major  terrain  features  change  little  over  time.  Identify  the  major 
terrain  feature,  highlighted  on  the  overhead  image,  on  which  the  road  junction  is 
located.  (Circle  the  correct  answer.) 

[Question  9  =  41 .8%  correct] 


MAP  VIEW 


OVERHEAD  IMAGE  VIEW 


A.  Hill 

D.  Valley 

B.  Ridge 

E.  Depression 

C.  Saddle 
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10.  The  current  rules  of  engagement  require  that  targets  engaged  with  indirect  fires 
must  be  observed  targets,  more  than  250  meters  away  from  the  civilian  population  or 
inhabited  structures,  and  located  to  within  ±  10  meters.  The  3rd  Squad  is  in  contact  but 
unable  to  communicate  with  the  platoon  leader.  They  are  pinned  down  by  RPG  and 
machine  gun  fire  coming  from  a  cluster  of  trees  on  a  hill  side,  well  outside  (±  800m)  the 
nearest  village.  You  alert  the  platoon  leader  and  inform  him  that  you  will  relay  their  Call 
for  Fire.  As  you  relay  the  target,  which  grid  location  provides  the  minimum  information 
required  for  target  approval?(Circle  the  correct  answer.) 

[Question  10  =  56.7%  correct] 

A.  16S  GA  04  85 

B.  16S  GA  046  854 

C.  16S  GA  0468  8542 

D.  16S  GA  04688  85422 

E.  N  32°  23’  W  084°  49’ 

1 1 .  You  are  training  your  Soldiers  to  navigate  with  a  map  and  compass  across  broken 
terrain.  You  want  them  to  be  aware  of  some  inaccuracies  with  their  navigation 
techniques.  For  example,  you’re  Alpha  Team  Leader  can  follow  an  azimuth  to  an 
accuracy  of  ±  3°  when  traveling  a  distance  of  1000  meters  or  less.  If  he  departs  from  a 
known  point  and  navigates  toward  a  known  point  1000  meters  away,  how  close  should 
he  be  to  his  desired  destination  point  on  arrival?  (Circle  the  best  answer.) 

[Question  11  =  11.9%  correct] 

A.  Within  150  meters 

B.  Within  110  meters 

C.  Within  55  meters 

D.  Within  35  meters 

E.  Within  20  meters 
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12.  Which  grid  coordinate  correctly  locates  the  trail  intersection  in  the  circle  to  within  10 
meters?  (Circle  one  answer.) 

[Question  12  =  89.6%  correct] 


A.  16SGA857  046 

B.  16S  GA  046  857 

C.  16S  GA  8575  0465 

D.  16S  GA  0465  8575 
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13.  What  feature  is  located  at  grid  16S  GL  0226  8538?  (Circle  the  answer.) 

[Question  13  =  73.1%  correct] 


A.  Mine 

B.  Borrow  pit 

C.  Hill  top 

D.  Road  intersection 

E.  Railroad  siding 

F.  Railroad  overpass 
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14.  It  is  summer  time,  your  current  location  is  indicated  by  the  point  marked  “P”  in  the 
map  and  photo  below.  Which  photograph  depicts  your  view  from  that  location  looking  to 
the  South?  (Circle  the  best  answer.) 

[Question  14  =  71.6%  correct] 
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15.  Your  squad  is  the  lead  element  in  your  company’s  air  assault  mission.  While  your 
helicopter  was  to  approach  the  LZ  from  the  West,  ground  fires  and  sightings  of  an 
enemy  mounted  patrol  caused  the  aircraft  to  take  evasive  action.  After  landing  you 
need  to  quickly  but  accurately  orient  your  map  and  prepare  for  the  arrival  of  the  main 
body  in  the  next  assault  wave.  Which  method  indicated  below  IS  NOT  a  correct  method 
of  map  orientation  with  a  compass?  (Circle  the  answer  that  best  identifies  the  incorrect 
method  or  methods  of  map  orientation.) 

[Question  15  =  14.9%  correct] 

A.  Place  the  map  on  a  flat  level  surface,  align  the  left  (straight)  edge  of  the  compass 
with  left  or  right  printed  edge  of  the  map.  Turn  the  map  till  the  magnetic  North  of  the 
compass  is  pointing  to  the  top  of  the  map  and  aligns  with  or  is  parallel  to  the  straight 
edge  of  the  compass. 

B.  Place  the  map  on  a  flat  level  surface,  then  align  the  north  indicating  line  on  the  map 
(in  the  margin  data)  with  the  magnetic  North  of  the  compass. 

C.  Place  the  map  on  a  flat  level  surface.  Find  the  magnetic  declination  data  in  the  map 
margin.  Place  the  compass  on  the  map  with  the  left  (straight)  edge  along  a  North-South 
Meridian  (longitudinal  line).  Turn  the  map  and  compass  together  until  the  N  arrow  on 
the  compass  is  offset  to  the  degree  angle  specified  by  the  declination  data. 

D.  If  your  location  is  known,  select  a  prominent  terrain  feature  visible  from  your 
location.  Sight  an  azimuth  from  your  position  to  the  terrain  feature.  Position  the  open 
compass  so  that  the  left  (straight)  edge  is  on  or  parallel  to  a  line  from  your  position  to 
the  terrain  feature.  Turn  the  map  and  compass  together  until  the  sight  azimuth  is 
aligned  on  the  compass. 

E.  None  of  the  above  techniques  are  correct  for  orienting  a  map  with  a  compass. 
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16.  While  moving  in  the  desert  you  cross  a  railroad  track,  to  determine  your  location 
you:  (Select  the  best  answer.) 

[Question  16  =  67.2%  correct] 

A.  Sight  an  azimuth  to  a  prominent  terrain  feature  and  determine  where  it  intersects 
along  the  railroad. 

B.  Sight  an  azimuth  to  two  prominent  terrain  features  and  determine  the  point  of 
intersection  on  the  map. 

C.  Both  A  and  B  are  correct. 

D.  Neither  A  or  B  is  correct. 

17.  When  navigating  at  night  with  a  lensatic  compass,  which  technique  would  NOT  be 
a  navigation  technique  to  consider?  (Circle  the  answer  that  best  identifies  the  incorrect 
method  or  methods  for  night  navigation.) 

[Question  17  =  44.8%  correct] 

A.  Sight  the  desired  azimuth  and  select  a  far  away  object  or  steering  point  with  a  distinct 
silhouette  against  the  sky  to  travel  toward. 

B.  In  dense  terrain,  select  steering  marks  that  are  closer  together  but  have  a  distinct 
silhouette  or  a  unique  appearance  to  travel  toward. 

C.  When  steering  marks  are  not  visible,  hold  the  fully  open  compass  with  the  desired 
travel  azimuth  in  front  of  your  chest,  and  periodically  check  your  azimuth  as  you  travel. 

D.  Walk  while  continually  observing  the  compass  dial  and  azimuth  and  avoid  taking 
sightings  as  you  travel. 

E.  None  of  these  techniques  are  recommended  for  night  navigation. 
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18.  Your  squad  is  leading  a  night  tactical  movement  of  your  platoon.  The  platoon 
leader  approved  your  route  around  a  village  to  avoid  observation.  Traveling  on  an 
azimuth  of  76°  you  arrived  at  Waypoint  4  (18S  WK  3764  2754).  You  must  now  turn  to 
an  azimuth  of  346°  toward  Waypoint  5  (18S  WK  3732  2887).  You  had  indexed  the 
bezel  ring  of  your  lensatic  compass  to  allow  travel  along  the  76°  azimuth.  What 
direction  and  how  many  clicks  should  you  rotate  the  bezel  to  align  the  compass  to 
346°?  (Circle  the  correct  answer.) 

[Question  18  =  3%  correct] 

A.  23  clicks  counter-clockwise 

B.  30  clicks  counter-clockwise 

C.  90  clicks  counter-clockwise 

D.  90  clicks  clockwise 

E.  30  clicks  clockwise 

F.  23  clicks  clockwise 

G.  None  of  the  above  is  correct. 
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NOTE:  Questions  19  and  20  are  to  be  answered  without  access  to  a  map. 

19.  Your  unit  has  the  mission  to  link  up  with  a  friendly  militiaman.  He  will  guide  you  to 
an  insurgent  cache  of  weapons,  ammunition,  and  bomb  making  materials.  The  starting 
location  for  your  patrol  is  the  road  intersection  at  grid  52S  CG  0450  8701 .  The  link  up 
point  is  for  the  friendly  militiaman  is  52S  CG  0751  8402.  SGT  Smith,  leader  of  your 
Alpha  Fire  Team,  will  be  responsible  for  navigation  for  this  mission.  During  PCIs,  you 
want  to  check  his  plan.  What  should  SGT  Smith’s  approximate  grid  azimuth  and 
movement  distance  calculations  be  for  the  most  direct  route  to  the  link  up  point?  (Circle 
one  answer.) 

[Question  19  =  19.4%  correct] 

A.  1 60°  for  3750  meters 

B.  235°  for  3750  meters 

C.  1 60°  for  4250  meters 

D.  1 35°  for  4250  meters 

E.  1 35°  for  3750  meters 

G.  None  of  the  above  is  correct. 


D-14 


20.  Your  platoon  is  occupying  a  patrol  base  at  16S  GA  063848  in  a  dense  swampy 
area,  preparing  for  reconnaissance  missions.  Your  security  responsibilities  for  the 
patrol  base  include  manning  an  OP/LP  located  at  16S  GA  06495  84842.  Suddenly  you 
hear  trucks  and  tracked  vehicle  movement  and  your  OP/LP  calls  with  a  report.  The  OP 
reports  observing  dust  and  hearing  heavy  trucks  and  tracked  vehicles  approach  from 
the  Northwest  and  stop.  They  have  observed  dust  above  the  trees  on  an  azimuth  of 
325°  approximately  900  meters  from  the  OP.  What  is  the  approximate  grid  for  this 
sighting?  (Circle  one  answer.) 

[Question  20  =  14.9%  correct] 

A.  16S  GA  077  853 

B.  16SGA059  855 

C.  16SGA072  855 

D.  16SGA054  850 

E.  16SGA057  840 

F.  None  of  the  above  is  correct. 
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Appendix  E 

TLP  Prior  Knowledge  Test 

General  Instructions:  These  questions  will  not  be  used  for  academic  evaluations  in  ALC. 

They  will  only  be  used  to  assess  your  prior  knowledge  and  skills.  If  you  are  uncertain  of  the 
correct  answer,  leave  it  blank.  Record  only  those  answers  that  you  believe  are  correct. 

GENERAL  SITUATION:  You  have  just  received  a  message  from  your  platoon  leader. 
The  company  will  be  conducting  an  attack  in  the  morning  to  seize  the  village  of  Dlubac 
and  destroy  or  capture  insurgent  forces  holding  the  village.  You  pass  along  the  warning 
order  (WARNO)  to  your  fire  team  leaders  and  review  your  procedures  and  SOPs  in 
preparation  for  developing  your  order  and  executing  your  assigned  mission  and  tasks. 

Section  I.  Troop  Leading  Procedures  (TLP) 

The  steps  of  TLP  are: 

Receive  the  Mission 
Issue  a  Warning  Order 
Make  a  Tentative  Plan 
Initiate  Movement 
Conduct  Reconnaissance 
Complete  the  Plan 
Issue  the  Operations  Order 
Supervise  and  Refine 

1 .  Which  statement  below  best  describes  Troop  Leading  Procedures  (TLP)?  (Select 
one  answer.) 

[Question  1  =  89.2%  correct] 

A.  TLP  provide  a  set  of  rules  and  steps  that  are  to  be  followed  when  an  order  is 
received  to  assure  compliance  with  all  specified  and  implied  tasks  in  the  order. 

B.  TLP  provide  a  doctrinal  set  of  steps  and  procedures  to  be  followed  by  inexperienced 
leaders  or  when  units  do  not  have  an  SOP.  TLP  can  be  ignored  by  units  with  good 
SOPs  or  leaders  with  combat  experience. 

C.  TLP  provide  a  sequence  of  steps  and  actions  that  guide  and  assist  the  small  unit 
leader  in  using  the  time  and  resources  available  to  prepare  and  issue  orders  and 
execute  tactical  operations. 

D.  Each  step  of  TLP  should  be  carefully  considered,  while  it  may  be  omitted  or 
eliminated,  it  should  only  be  executed  once  in  the  sequence. 

E.  The  concept  of  TLP  is  under  revision  and  is  being  replaced  at  all  echelons  by  the 
Military  Decision  Making  Process  (MDMP). 


E-l 


2.  Rehearsals  may  be  conducted  as  an  aspect  of  the  Supervise  and  Refine  step  of 
TLP.  Rehearsals  may  be  simple  map  rehearsals  with  leaders,  sand  table  or  terrain 
model  rehearsals  that  permit  participants  to  observe  their  actions  and  tasks  in  relation  to 
others,  or  reduced-  or  full-force  rehearsals  to  allow  movements  and  actions  under 
conditions  similar  to  those  expected  in  the  AO. 

[Question  2  =  91.9%  correct] 

Which  actions  listed  below  CAN  NOT  be  accomplished  by  well-planned  rehearsals? 
(Select  one.) 

A.  Reveal  weaknesses  or  problems  in  the  plan. 

B.  Eliminate  all  uncertainties  about  how  the  enemy  will  act  or  react. 

C.  Reinforce  training  readiness  and  increase  proficiency  in  critical  tasks. 

D.  Review  and  practice  contingencies  and  “be  prepared”  tasks  of  the  mission. 

E.  Confirm  leader  and  Soldier  understanding  of  the  mission  and  tasks  to  be  performed. 


3.  TLP  may  be  conducted  in  a  time  constrained  environment.  Some  steps  of  TLP  may 
be  considered  and  omitted.  However,  other  steps  may  recur  several  times  during 
planning.  Identify  the  step  or  steps  that  may  occur  several  times  or  be  continuous 
during  TLP.  (Select  all  that  apply.) 

[Question  3a  =  32.4%,  3b  =  17.6%,  3c  =  40.5%,  3d  =  74.3%,  3e  =  87.8%  correct] 

A.  Initiate  movement  or  relocate  to  facilitate  mission  planning,  preparation,  or 
execution. 

B.  Issue  a  warning  order. 

C.  Conduct  reconnaissance 

D.  Supervise  and  refine  plans 

E.  None  of  the  above  elements  are  recurring  or  continuous. 
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4.  Dissemination  of  WARNOs  can  support  mission  planning  and  preparation.  What  are 
some  of  the  ways  that  WARNOs  can  have  a  positive  impact  on  TLP?  (Select  all  that 
apply.) 

[Question  4a  =  74.3%,  4b  =  56.8%,  4c  =  56.8%,  4d  =  81.1%,  4e  =  54.1%  correct] 

A.  Alert  subordinates  to  a  new  or  changing  mission. 

B.  Provide  information  on  the  time  available  to  prepare. 

C.  Provide  information  on  the  enemy,  mission  tasks,  and  guidance  for  preparations. 

D.  List  all  courses  of  action  being  considered. 

E.  Permit  preparations  to  take  place  in  parallel  with  planning. 
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Section  II.  Operations  Orders. 

Use  the  order  format  below  to  assist  in  answering  questions  5-13;  see  the  example 
below. 

You  have  received  an  order  from  your  platoon  leader.  Initially,  your  squad  is  the  main 
effort.  He  asks  you  to  inform  him  of  the  time  that  you  will  issue  your  oral  order  to  your 
fire  team  leaders  or  the  squad.  The  platoon  leader  and  company  commander  will  be 
present  when  you  brief  your  order.  Squad  orders  are  generally  direct  and  simple.  Use 
the  five-paragraph  operations  order  (OPORD)  format  outlined  below  to  organize  the 
information  in  your  orders  briefing: 


OPORD  Reference 

Short 
Answer  Kev 

Five-paraqraph  Order  Example 

A 

1.  Situation. 

B 

a.  Enemy  Forces. 

C 

b.  Friendly  Forces. 

D 

c.  Attachments  and  Detachments. 

E 

2.  Mission. 

F 

3.  Execution. 

G 

a.  Concept  of  Operations. 

H 

b.  Maneuver. 

1 

c.  Fires. 

J 

4.  Service  Support. 

K 

5.  Command  and  Signal. 

L 

a.  Command. 

M 

b.  Signal. 
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Example  question: 

For  the  next  mission,  each  fire  team  will  be  issued  an  M26  Modular  Accessory  Shotgun 
System  (MASS)  and  30  rounds  of  door  breaching  (frangible  slugs)  ammunition.  Where 
should  this  information  be  inserted  in  the  order? 

Example  Answer: _ J _ 


Remember,  you  will  brief  your  order  to  your  squad.  Use  the  five-paragraph  format  to 
help  organize  the  information. 

5.  A  machine  gun  team  from  the  Platoon’s  Weapons  Squad  and  an  engineer  demolition 
team  will  be  attached  for  the  next  mission.  Where  in  the  order  should  this  addition  to 
your  squad  be  indicated? 

[Question  5  =  78.4%  correct] 

Answer: _ 

6.  The  Platoon  Sergeant  will  establish  an  initial  casualty  collection  point  (CCP)  on  the 
North  side  of  Building  H  at  1 6S  GA  0651  8363.  He  plans  to  move  the  CCP  near  the 
breach  point  after  entry.  Where  should  CCP  information  be  indicated  in  your  order? 

[Question  6  =  45.9%  correct] 

Answer: _ 

7.  After  a  brief  review  of  your  Platoon  Leader’s  order,  you  have  developed  the 
following  statement,  “3rd  Squad  attacks  tomorrow  after  0400  on  signal  to  seize  the 
northwest  wall  of  Building  A2  (16S  GA  0653  8359)  and  create  a  breach,  pass  the 
platoon  (-)  and  support  the  clearing  and  seizure  of  Building  A2.”  Where  should  this 
statement  appear  in  your  order? 

[Question  7  =  43.2%  correct] 

Answer:  _ 

8.  Your  Platoon  (1st)  will  be  the  main  effort  passing  through  3rd  Platoon  and  supported 
by  an  attack  by  2nd  Platoon.  Where  should  you  discuss  the  mission  and  locations  of  the 
2nd  and  3rd  Platoons? 

[Question  8  =  47.3%  correct] 

Answer: 
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9.  You  have  information  on  the  location  of  two  insurgent  snipers  and  some  details  on 
the  locations  and  dispositions  of  insurgent  forces  in  the  Platoon’s  Objective  Bone 
(Building  C4).  Where  should  this  information  go  in  the  order? 

[Question  9  =  86.5%  correct] 

Answer: 


10.  The  weather  is  expected  to  change  tonight  from  the  current  mild  temperatures  and 
partly  cloudy  skies.  Rain,  low  clouds,  and  fog  are  expected  after  midnight.  In  which 
major  paragraph  of  the  order  is  weather  usually  discussed? 

[Question  10  =  55.4%  correct] 

Answer: 


11.  In  which  section  of  the  order  should  you  indicate  the  specific  tasks  to  be  performed 
by  your  fire  teams  and  attached  elements? 

[Question  11=  33.8%  correct] 

Answer:  _ 

12.  In  addition  to  radio  communications,  a  green  star  cluster  will  be  used  to  order  the 
attack  to  commence.  Purple  smoke  will  be  used  to  mark  the  entry  hole  when  the 
breach  is  made.  Where  should  this  information  be  indicated  in  the  order? 

[Question  12  =  66.2%  correct] 

Answer: 


13.  The  platoon  leader  will  initially  move  with  2nd  Squad  (immediately  behind  you)  and 
move  through  the  breach  that  you  will  create.  What  section  of  the  order  should  contain 
information  on  the  location  of  the  Platoon  Headquarters? 

[Question  13  =  17.6%  correct] 

Answer: 
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Section  III.  Military  Symbols  and  Graphics. 

The  Platoon  Leader  provided  you  this  map  and  diagram  when  he  issued  his  operations  order.  Use  the  operation’s  overlay 
on  the  photo  map  and  the  operation’s  sketch  to  assist  in  answering  questions  14-20. 
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*  <S> 

vi  u 


Enter  the  letter  that  points  to  the  operations  symbol.  (Note:  Not  all  symbols  or  letters 
may  match.) 

14.  The  boundary  between  2nd  BN  4th  INF  and  3rd  SQDN  89th  CAV  _ 

[Question  14  =  67.2%  correct] 

15.  The  forward  line  of  troops  or  front  line  trace  _ 

[Question  15  =  68.9%  correct] 

16.  A  tactical  objective  in  the  C  Company,  2nd  BN  4th  INF’s  sector  _ 

[Question  16  =  56.8%  correct] 

17.  The  boundary  between  2nd  BN  4th  INF  and  2nd  BN  30th  INF  _ 

[Question  17  =  79.7%  correct] 


18.  Snipers  have  been  detected  in  the  village.  While  the  enemy  may  move,  in  which 
buildings  have  snipers  been  confirmed?  (Place  a  0  or  0  in  the  box  in  front  of  the 
building  identifier.) 

[Question  18a1  =  82.4%,  18a2  =  66.2%,  18b1  =  63.5%,  18c  =  63.5%,  18h  =  63.5%, 
18p4  =  56.8%,  18p5  =  70.3%  correct] 


□ 

A1 

□ 

A2 

□ 

B1 

□ 

C 

□ 

H 

□ 

P4 

□ 

P5 
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19.  What  size  and  type  element  is  being  reported  in  your  objective  -  Objective  Bone, 
Building  A2?  (Circle  the  letter.) 

[Question  19  =  51.4%  correct] 

A.  Infantry  Fire  Team 

B.  Infantry  Squad 

C.  Medical  Squad 

D.  Engineer  Squad 

E.  Infantry  Platoon 

20.  As  you  examine  the  situation,  you  determine  that  you  could  receive  effective  fires 
from  the  squad  sized  insurgent  element  less  than  200  meters  to  your  East.  Which  unit 
is  responsible  to  neutralize  this  threat? 

[Question  20  =  45.9%  correct] 

A.  2nd  Platoon,  A,  2-4  INF 

B.  3rd  Platoon,  C,  2-4  INF 

C.  C,  2-4  INF 

D.  C,  3-89  CAV 

E.  2-30  INF 
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Appendix  F 
Descriptive  Statistics 


Table  F-l 
Prior  MOS 


Value 

Frequency 

Percent 

Yes 

13 

17.8 

No 

60 

82.2 

Total 

73 

100 

Table  F-2 

Civilian  Education  Level 


Value 

Frequency 

Percent 

GED 

5 

6.8 

HS  Diploma 

26 

35.1 

Some  College 

36 

48.6 

Associates 

4 

5.4 

Bachelors 

3 

4.1 

Total 

74 

100 

Note.  College  degrees  specified  were  as  fo 


History,  and  Hotel  Management. 


lows  (1  of  each):  Business,  Criminal  Justice,  General, 
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Table  F-3 

General  Land  Navigation  Experience  Questions 


Experience  with  Global  Positioning  Systems 

Value 

Number  of  Soldiers 

Percent  Soldiers 

Yes 

73 

98.6 

No 

1 

1.4 

Total 

74 

100 

If  GPS  Experience,  Also  Use  ol 

'Maps  and  Compass 

Value 

Frequency 

Percent 

Yes 

61 

83.6 

No 

12 

16.4 

Total 

73 

100 

If  Use  of  Maps/Compass,  Used  as  1 

*rimary  Navigation  Aids 

Value 

Frequency 

Percent 

Yes 

18 

27.7 

No 

47 

72.3 

Total 

65 

100 

Used  FBCB2,  Land  Warrior,  Other  C2  System 

Value 

Frequency 

Percent 

Yes 

72 

97.3 

No 

2 

2.7 

Total 

74 

100 

Table  F-4 

Frequency  Scale  Summary  Score  Statistics 


Statistic 

Land  Navigation 
Frequency  Scale 

TLP  Frequency 
Scale 

TLP  Frequency 
Scale  2 

Mean 

31.56 

26.49 

11.66 

SD 

9.55 

6.33 

2.28 

Minimum 

11 

10 

6 

Maximum 

53 

35 

15 
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Table  F-5 

Land  Navigation  Prior  Knowledge  Test  Item  Statistics 


Question 

Number 

Mean 

SD 

Question 

Number 

Mean 

SD 

1 

.64 

.48 

10 

.57 

.50 

2a 

.45 

.50 

11 

.12 

.33 

2b 

.69 

.47 

12 

.90 

.31 

2c 

.54 

.50 

13 

.73 

.45 

2d 

.07 

.26 

14 

.72 

.45 

3 

.66 

.48 

15 

.15 

.36 

4 

.84 

.37 

16 

.67 

.47 

5 

.76 

.43 

17 

.45 

.50 

6 

.08 

.26 

18 

.03 

.17 

7 

.24 

.43 

19 

.19 

.40 

8 

.36 

.48 

20 

.15 

.36 

9 

.42 

.50 

Note.  N=  67  respondents. 

Mean  =  Percent  correct  responses. 


Table  F-6 

Land  Navigation  Prior  Knowledge  Test  Summary  Score  Statistics 


Variable 

Statistic 

Mean 

SD 

Min 

Max 

Total  Points 

10.40 

3.04 

4 

18 

Percent  Correct 

45 

13.20 

17 

78 
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Table  F-7 

TLP  Prior  Knowledge  Test  Item  Statistics 


Question 

Number 

Mean 

SD 

Question 

Number 

Mean 

SD 

1 

.81 

.39 

10 

.55 

.50 

2 

.92 

.27 

11 

.34 

.48 

3a 

.32 

.47 

12 

.66 

.48 

3b 

.18 

.38 

13 

.18 

.38 

3c 

.41 

.49 

14 

.62 

.49 

3d 

.74 

.44 

15 

.69 

.47 

3e 

.89 

.31 

16 

.57 

.50 

4a 

.74 

.44 

17 

.80 

.40 

4b 

.57 

.50 

18a 

.86 

.34 

4c 

.57 

.50 

18b 

.66 

.48 

4d 

.82 

.38 

18c 

.64 

.48 

4e 

.54 

.50 

18d 

.64 

.48 

5 

.78 

.41 

18e 

.64 

.48 

6 

.46 

.50 

18f 

.57 

.50 

7 

.43 

.50 

18g 

.70 

.46 

8 

.47 

.50 

19 

.51 

.50 

9 

.88 

.33 

20 

.46 

.50 

Note.  N  =  74  respondents. 

Mean  =  Percent  correct  responses. 


Table  F-8 


TLP  Prior  Knowledge  Test  Summary  Score  Statistics 


Variable 

Statistic 

Mean 

SD 

Min 

Max 

Total  Points 

20.62 

5.72 

6 

32 

Percent  Correct 

61 

17 

17 

94 
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Table  F-9 

Land  Navigation  Criterion  Summary  Score  Statistics 


Variable 

Statistic 

Mean 

SD 

Min 

Max 

Total  Points 

5.33 

1.47 

0 

8 

Percent  Correct 

67 

19 

0 

100 

Note.  V=67  respondents. 


Table  F-10 

Land  Navigation  Criterion  Go/No  Go  Status 


Value 

Number  of 

Soldiers 

Percent 

No  Go 

9 

13.4 

Go 

58 

86.6 

Total 

67 

100 

Table  F-ll 

TLP  Criterion  Summary  Score  Statistics 


Variable 

Statistic 

Mean 

SD 

Min 

Max 

Total  Points 

24.57 

3.24 

15 

30 

Percent  Correct 

82 

11 

50 

100 

Note.  N=72  respondents. 
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